Hilal Nizamoğlu^1,2, Katharina Dobs^1,2; ¹Justus Liebig University in Giessen, Germany, ²Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen

Previous studies on dynamic faces have shown that both rigid and non-rigid facial movements contribute to identity recognition, and that the identity-specific information inherent in these motions varies depending on the type of facial expression. Therefore, it becomes clear that individuals exhibit distinct idiosyncratic patterns in their facial movements, which can serve as cues towards their identity. However, the specific features of facial movements that contribute to this uniqueness remain unclear. Here, we employed machine learning techniques to measure and quantify motion information in facial expressions, using a dataset of six basic emotional facial expressions (anger, disgust, fear, joy, sad, surprise) performed by 12 German and 12 Turkish lay actors. An automated facial landmark detection tool was applied to measure the positional changes of landmarks at the peak of each expression relative to a neutral baseline. We then trained a Linear Discriminant Analysis (LDA) model with these landmark shifts to classify the emotional expressions. This first LDA model was able to classify the type of emotional expression (Accuracy: 44%, p<0.001), independent of the actor’s identity. More strikingly, another LDA model, trained to classify the identities of the 24 actors across different expressions, successfully predicted their identity (Accuracy: 45%, p<0.001). Furthermore, the landmark positional changes provided useful information for classifying actors’ gender (Accuracy: 59%, p<0.01) and country-of-origin (Accuracy: 71%, p<0.001), supporting previous studies on cultural and gender-based variations in facial expressions. In conclusion, our study shows the richness of information embedded in facial motion features, extending beyond emotional expression to contain aspects of the actor’s identity, gender, and cultural background. This landmark-based approach emerges as a promising tool to unravel the nuances of idiosyncrasies in facial movements, offering valuable insights into the intricate interplay of expression, identity, and cultural factors.

Acknowledgements: This work was supported by DFG, Germany SFB/TRR 135 (grant number 222641018) TP C9 and S; and by the Research Cluster ‘‘The Adaptive Mind’’, funded by the Hessian Ministry for Higher Education, Research Science and the Arts. We also thank Prof. Dr. Benjamin Straube for the video dataset.