• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学 ›› 2022, Vol. 44 ›› Issue (11): 2027-2036.

• 图形与图像 • 上一篇    下一篇

带有协方差矩阵的卷积神经网络在人体运动识别中的应用

权威铭,刘天一,张雷   

  1. (南京师范大学电气与自动化工程学院,江苏 南京 210046)
  • 收稿日期:2021-03-30 修回日期:2021-05-14 接受日期:2022-11-25 出版日期:2022-11-25 发布日期:2022-11-25
  • 基金资助:
    国家自然科学基金(61203237);江苏省自然科学基金(BK20191371) 

Application of convolutional neural network with covariance matrix in human activity recognition

QUAN Wei-ming,LIU Tian-yi,ZHANG Lei   

  1. (School of Electrical and Automation Engineering,Nanjing Normal University,Nanjing 210046,China)
  • Received:2021-03-30 Revised:2021-05-14 Accepted:2022-11-25 Online:2022-11-25 Published:2022-11-25

摘要: 目前,深度学习已经在各种人体运动识别(HAR)任务中发挥了重要作用。但是,由于运动数据具有时间序列和包含肢体动作的特殊性,现有神经网络在进行卷积操作时会导致数据高度相关,并且随着网络影响到下一层,这限制了模型的识别效果。为此,提出了一种带有协方差矩阵的改进卷积神经网络用于HAR场景,通过矩阵变换搭建一种去相关的网络结构来消除相关性问题,可以在网络表现不佳时替代现有的批量归一化(BN)层用于归一化数据。在4个HAR公共数据集上进行实验,并与传统CNN和带有BN层的模型进行比较。实验结果表明,对比此前的深度学习网络,改进的神经网络有1%~2%的性能提升,验证了该方法的有效性,并将程序移植到了移动端进行实时运动识别。

关键词: 可穿戴传感器, 运动识别, 卷积神经网络, 协方差矩阵

Abstract: At present,deep learning has played an important role in various human activity recognition (HAR) tasks.However,the activity data has the particularity of time series and includes body movements.The existing convolutional neural network (CNN) will cause the data to be highly correlated when performing convolutional operations. As the network affects the next layer, the accuracy of network recognition is limited. In order to solve this phenomenon, this paper proposes an improved convolutional neural network with covariance matrix for HAR scenario. It builds a de-correlated network structure through matrix transformation to eliminate correlation problems. When the network performance is poor, the network can replace the existing BN layer to normalize data. The verification experiments are finished on four HAR public datasets. The proposed neural network is compared with traditional CNN model and BN layer model. The results show that the improved neural network is improved by 1% to 2% compared with the previous deep learning networks, which proves that the improved neural network is effective. Furthermore, the application is transplanted to the mobile terminal for real-time activity recognition. 

Key words: wearable sensor, activity recognition, convolutional neural network, covariance matrix