Space-by-time manifold representation of dynamic facial

Delis, I., Chen, C., Jack, R. E. , Garrod, O. G.B., Panzeri, S. and Schyns, P. G. (2016) Space-by-time manifold representation of dynamic facial. Journal of Vision, 16(8), pp. 1-20. 14. (doi: 10.1167/16.8.14) (PMID:27305521) (PMCID:PMC4927208)

119969.pdf - Published Version
Available under License Creative Commons Attribution.



Visual categorization is the brain computation that reduces high-dimensional information in the visual environment into a smaller set of meaningful categories. An important problem in visual neuroscience is to identify the visual information that the brain must represent and then use to categorize visual inputs. Here we introduce a new mathematical formalism—termed space-by-time manifold decomposition—that describes this information as a low-dimensional manifold separable in space and time. We use this decomposition to characterize the representations used by observers to categorize the six classic facial expressions of emotion (happy, surprise, fear, disgust, anger, and sad). By means of a Generative Face Grammar, we presented random dynamic facial movements on each experimental trial and used subjective human perception to identify the facial movements that correlate with each emotion category. When the random movements projected onto the categorization manifold region corresponding to one of the emotion categories, observers categorized the stimulus accordingly; otherwise they selected “other.” Using this information, we determined both the Action Unit and temporal components whose linear combinations lead to reliable categorization of each emotion. In a validation experiment, we confirmed the psychological validity of the resulting space-by-time manifold representation. Finally, we demonstrated the importance of temporal sequencing for accurate emotion categorization and identified the temporal dynamics of Action Unit components that cause typical confusions between specific emotions (e.g., fear and surprise) as well as those resolving these confusions.

Item Type:Articles
Additional Information:This work was supported by the Economic and Social Research Council (ESRC ES/K001973/1), the Wellcome Trust (107802/Z/15/Z), and the European Commission (FP7-600954, ‘‘VISUALISE’’).
Glasgow Author(s) Enlighten ID:Jack, Professor Rachael and Garrod, Dr Oliver and Chen, Dr Chaona and Delis, Dr Ioannis and Panzeri, Professor Stefano and Schyns, Professor Philippe
Authors: Delis, I., Chen, C., Jack, R. E., Garrod, O. G.B., Panzeri, S., and Schyns, P. G.
College/School:College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
Journal Name:Journal of Vision
Publisher:Association for Research in Vision and Ophthalmology
ISSN (Online):1534-7362
Published Online:15 June 2016
Copyright Holders:Copyright © 2016 The Authors
First Published:First published in Journal of Vision 16(8):14, 1-20
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
698281Brain Algorithmics: Reverse Engineering Dynamic Information Processing Networks from MEG time seriesPhilippe SchynsWellcome Trust (WELLCOME)107802/Z/15/ZINP - CENTRE FOR COGNITIVE NEUROIMAGING
590701Mapping the Cultural Landscape of Emotions for Social InteractionRachael JackEconomic & Social Research Council (ESRC)ES/K001973/1S&E PSY - ADMINISTRATION