Revealing concealed spontaneous facial micro-expression: Are we a step closer to unveil real-life behavioral expressions?

• Main Authors: Gan, Y. S., Liong, Gen Bing, Liu, Kun-Hong, Liong, Sze-Teng
• Format: Article
• Published: Elsevier 2023
• Subjects: QA75 Electronic computers. Computer science
• Online Access: http://eprints.um.edu.my/38293/

Facial micro-expression (ME) expression analysis has aroused interdisciplinary attention from the realms of computer vision and psychology due to its advantages of divulging nonverbal emotion via subtle invol-untary facial muscle changes. However, developing and commercializing ME recognition relevant prod-ucts has been hindered in current progress due to the scarcity of databases depicting real-world conditions. This paper aims to give a strong impetus to facilitating the advancement of ME recognition systems, particularly in in-the-wild applications. Succinctly, an efficient recognition system is introduced herein, which incorporates the core processes like 3D facial reconstruction, apex spotting, and emotion recognition tasks. Concretely, all faces are first rendered into 3D point cloud form for the brevity of face alignment along the video. Then, the optical flow-guided components are employed as the primary fea-tures to represent the motion details. Subsequently, a 3-Stream channel based on 3-Dimensional faces Network (3T3D-Net) with skip connection via multiplication is tailored to cope with the small-size input image. As a result, pleasing recognition performance is yielded via the suite of novel techniques devised, by producing a UAR of 69.44% and UF1 of 70.71%, when evaluated on the in-the-wild ME dataset, viz, MEVIEW. In addition, extensive studies and analyses are presented to verify the contribution of each component of the proposed pipeline. It is envisioned that the remarkable result attained by casting an invigorating perspective will shed light and pave a new prospect for analysis in ME applications. (c) 2023 Elsevier B.V. All rights reserved.