Audiovisual integration of emotional signals from music improvisation does not depend on temporal correspondence

Petrini, K., McAleer, P. and Pollick, F.E. (2010) Audiovisual integration of emotional signals from music improvisation does not depend on temporal correspondence. Brain Research, 1323, pp. 139-148. (doi: 10.1016/j.brainres.2010.02.012)

Full text not currently available from Enlighten.


In the present study we applied a paradigm often used in face–voice affect perception to solo music improvisation to examine how the emotional valence of sound and gesture are integrated when perceiving an emotion. Three brief excerpts expressing emotion produced by a drummer and three by a saxophonist were selected. From these bimodal congruent displays the audio-only, visual-only, and audiovisually incongruent conditions (obtained by combining the two signals both within and between instruments) were derived. In Experiment 1 twenty musical novices judged the perceived emotion and rated the strength of each emotion. The results indicate that sound dominated the visual signal in the perception of affective expression, though this was more evident for the saxophone. In Experiment 2 a further sixteen musical novices were asked to either pay attention to the musicians' movements or to the sound when judging the perceived emotions. The results showed no effect of visual information when judging the sound. On the contrary, when judging the emotional content of the visual information, a worsening in performance was obtained for the incongruent condition that combined different emotional auditory and visual information for the same instrument. The effect of emotionally discordant information thus became evident only when the auditory and visual signals belonged to the same categorical event despite their temporal mismatch. This suggests that the integration of emotional information may be reinforced by its semantic attributes but might be independent from temporal feature

Item Type:Articles
Glasgow Author(s) Enlighten ID:McAleer, Dr Phil and Pollick, Professor Frank
Authors: Petrini, K., McAleer, P., and Pollick, F.E.
College/School:College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
Journal Name:Brain Research
ISSN (Online):1872-6240
Published Online:11 February 2010

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
449881Social interaction - a cognitive-neurosciences approachSimon GarrodEconomic & Social Research Council (ESRC)ES/E020933/1Cognitive Neuroimaging & Neuroengineering Technologies