Abstract: At present,deep learning has played an important role in various human activity recognition (HAR) tasks.However,the activity data has the particularity of time series and includes body movements.The existing convolutional neural network (CNN) will cause the data to be highly correlated when performing convolutional operations. As the network affects the next layer, the accuracy of network recognition is limited. In order to solve this phenomenon, this paper proposes an improved convolutional neural network with covariance matrix for HAR scenario. It builds a de-correlated network structure through matrix transformation to eliminate correlation problems. When the network performance is poor, the network can replace the existing BN layer to normalize data. The verification experiments are finished on four HAR public datasets. The proposed neural network is compared with traditional CNN model and BN layer model. The results show that the improved neural network is improved by 1% to 2% compared with the previous deep learning networks, which proves that the improved neural network is effective. Furthermore, the application is transplanted to the mobile terminal for real-time activity recognition. 

Key words: wearable sensor, activity recognition, convolutional neural network, covariance matrix