Integrating model-based prediction and facial expressions in the perception of emotion

Yongsatianchot, N. and Marsella, S. (2016) Integrating model-based prediction and facial expressions in the perception of emotion. In: Steunebrink, B., Wang, P. and Goertzel, B. (eds.) Artificial General Intelligence: 9th International Conference, AGI 2016, New York, NY, USA, July 16-19, 2016, Proceedings. Series: Lecture Notes in Computer Science, 9782. Springer, pp. 234-243. ISBN 9783319416489 (doi: 10.1007/978-3-319-41649-6_23)

Full text not currently available from Enlighten.


Understanding a person’s mental state is a key challenge to the design of Artificial General Intelligence (AGI) that can interact with people. A range of technologies have been developed to infer a user’s emotional state from facial expressions. Such bottom-up approaches confront several problems, including that there are significant individual and cultural differences in how people display emotions. More fundamentally, in many applications we may want to know other mental states such as goals and beliefs that can be critical for effective interaction with a person. Instead of bottom-up processing of facial expressions, in this work, we take a predictive, Bayesian approach. An observer agent uses mental models of an observed agent’s goals to predict how the observed will react emotionally to an event. These predictions are then integrated with the observer’s perceptions of the observed agent’s expressions, as provided by a perceptual model of how the observed tends to display emotions. This integration provides the interpretation of the emotion displayed while also updating the observer’s mental and emotional display models of the observed. Thus perception, mental model and display model are integrated into a single process. We provide a simulation study to initially test the effectiveness of the approach and discuss future work in testing the approach in interactions with people.

Item Type:Book Sections
Glasgow Author(s) Enlighten ID:Marsella, Professor Stacy
Authors: Yongsatianchot, N., and Marsella, S.
College/School:College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience

University Staff: Request a correction | Enlighten Editors: Update this record