• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学 ›› 2023, Vol. 45 ›› Issue (05): 849-858.

• 图形与图像 • 上一篇    下一篇

融合局部动态特征的面部表情识别

刘南艳,魏鸿飞,马圣祥   

  1. (西安科技大学计算机科学与技术学院,陕西 西安 710699)
  • 收稿日期:2021-12-17 修回日期:2022-01-20 接受日期:2023-05-25 出版日期:2023-05-25 发布日期:2023-05-16
  • 基金资助:
    国家自然科学基金(62002285)

Facial expression recognition fusing local dynamic features

LIU Nan-yan,WEI Hong-fei,MA Sheng-xiang   

  1. (College of Computer Science & Technology,Xi’an University of Science and Technology,Xi’an  710699,China)
  • Received:2021-12-17 Revised:2022-01-20 Accepted:2023-05-25 Online:2023-05-25 Published:2023-05-16

摘要: 面部表情是人类表达情感最重要的方式之一。面部表情变化受多个面部器官和面部肌肉运动的影响。为了能有效提取局部动态特征和解决面部表情部分遮挡问题,提出一种简单有效的融合局部动态特征的深度学习网络,通过构建引导注意网络,利用检测到的脸部关键点来引导网络关注无遮挡的面部区域。为了强化不同表情特征之间的联系,利用局部动态特征网络,在带有时间序列的关键帧中提取如眼睛、嘴巴等关键区域的动态信息和时空信息。最后,将局部动态特征补充到整体网络中。融合后的网络在CK+、Oulu-CASIA、RAF-DB和AffectNet数据集上的精度分别为98.08%,90.59%,86.02%和61.28%,相较于其它网络的识别率均有提高。

关键词: 动态特征;面部遮挡;引导注意网络;时间序列 ,

Abstract: Facial expressions are one of the most important ways for humans to express emotions. Because facial expression changes are affected by the movement of multiple facial organs and facial muscles, in order to effectively extract local dynamic features and solve the problem of partial occlusion of facial expressions, a simple and effective deep learning network that integrates local dynamic features is proposed. By introducing the attention network and using the monitored key points of the face, the network is guided to focus on the unobstructed facial area. In the key frame with time sequence, the dynamic information and spatiotemporal information of key areas such as eyes and mouth are extracted to strengthen the connection between different expression features, so as to obtain effective local dynamic features. Finally, local dynamic features are added as a supplement to the overall network. The accuracy of the fusion network on the CK+, Oulu-CASIA, RAF-DB and AffectNet datasets are 98.08%, 90.59%, 86.02% and 61.28%, respectively, which is higher than other methods. 

Key words: dynamic feature, facial occlusion, guide attention network, time series