Abstract: Facial Expression Recognition (FER) is usually affected by the collected environment, regions, race, and other factors. In order to improve the generalization of FER methods, Unsupervised Domain Adaption Facial Expression Recognition (UDA-FER) algorithms have attracted more and more attentions. Existing UDA-FER algorithms generally suffer from two issues: (1) they care more about the  performance in the target domain, resulting in a sharp drop in the performance of the source domain after transferring from the source to the target domain; (2) They are just appropriate for the case of the single target domain. The UDA-FER methods will show terrible performance when applying it to multiple target domains directly. To solve the above issues, a Multi-Target Domain Facial Expression Recognition method based on class-wise feature constraint (MTD-FER) is proposed, which supports the FER methods transferring to multiple target domains in succession and ensures the methods retains a better recognition rate on each domain. To this end, MTD-FER designs the Class-Adaptive Pseudo Label methods (CAPL) and Class-Wise Feature Constraint mothods(CWFC), which learn pseudo labels for samples with high quality in target domains and align each class of features from disparate domains, so as to alleviate the issue of catastrophic forgetting resulting from domain transferring. Through extensive experiments using RAF-DB as the source domain and FER-2013 and ExpW as the target domains, the effectiveness of the MTD-FER algorithm is demonstrated. Experimental results show that, compared with the baseline method, MTD-FER improves the performance in the source domain by 6.36%, which is on par with the methods before transferring to target domains, and improves the performance by 27.33% and 3.03% in two target domains, respectively.

Key words: facial expression recognition, unsupervised domain adaptation, multi-target domain, class-adaptive pseudo label, class-wise feature constraint