Synthetic vs human emotional faces: what changes in humans’ decoding accuracy

Amorese, T., Cuciniello, M., Vinciarelli, A. , Cordasco, G. and Esposito, A. (2022) Synthetic vs human emotional faces: what changes in humans’ decoding accuracy. IEEE Transactions on Human-Machine Systems, 52(3), pp. 390-399. (doi: 10.1109/THMS.2021.3129714)

Abstract

Considered the increasing use of assistive technologies in the shape of virtual agents, it is necessary to investigate those factors which characterize and affect the interaction between the user and the agent, among these emerges the way in which people interpret and decode synthetic emotions, i.e., emotional expressions conveyed by virtual agents. For these reasons, an article is proposed, which involved 278 participants split in differently aged groups (young, middle-aged, and elders). Within each age group, some participants were administered a “naturalistic decoding task,” a recognition task of human emotional faces, while others were administered a “synthetic decoding task” namely emotional expressions conveyed by virtual agents. Participants were required to label pictures of female and male humans or virtual agents of different ages (young, middle-aged, and old) displaying static expressions of disgust, anger, sadness, fear, happiness, surprise, and neutrality. Results showed that young participants showed better recognition performances (compared to older groups) of anger, sadness, and neutrality, while female participants showed better recognition performances (compared to males) of sadness, fear, and neutrality; sadness and fear were better recognized when conveyed by real human faces, while happiness, surprise, and neutrality were better recognized when represented by virtual agents. Young faces were better decoded when expressing anger and surprise, middle-aged faces were better decoded when expressing sadness, fear, and happiness, while old faces were better decoded in the case of disgust; on average, female faces where better decoded compared to male ones.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Vinciarelli, Professor Alessandro
• Authors: Amorese, T., Cuciniello, M., Vinciarelli, A., Cordasco, G., and Esposito, A.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: IEEE Transactions on Human-Machine Systems
• Publisher: IEEE
• ISSN: 2168-2291
• ISSN (Online): 2168-2305
• Published Online: 08 December 2021