Contributions of local speech encoding and functional connectivity to audio-visual speech perception

Giordano, B. L., Ince, R. A.A., Gross, J., Schyns, P. G., Panzeri, S. and Kayser, C. (2017) Contributions of local speech encoding and functional connectivity to audio-visual speech perception. eLife, 6, e24763. (doi:10.7554/elife.24763) (PMID:28590903) (PMCID:PMC5462535)

[img]
Preview
Text
142077.pdf - Published Version
Available under License Creative Commons Attribution.

1MB

Abstract

Seeing a speaker’s face enhances speech intelligibility in adverse environments. We investigated the underlying network mechanisms by quantifying local speech representations and directed connectivity in MEG data obtained while human participants listened to speech of varying acoustic SNR and visual context. During high acoustic SNR speech encoding by temporally entrained brain activity was strong in temporal and inferior frontal cortex, while during low SNR strong entrainment emerged in premotor and superior frontal cortex. These changes in local encoding were accompanied by changes in directed connectivity along the ventral stream and the auditory-premotor axis. Importantly, the behavioral benefit arising from seeing the speaker’s face was not predicted by changes in local encoding but rather by enhanced functional connectivity between temporal and inferior frontal cortex. Our results demonstrate a role of auditory-frontal interactions in visual speech representations and suggest that functional connectivity along the ventral pathway facilitates speech comprehension in multisensory environments.

Item Type:Articles
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Kayser, Professor Christoph and Gross, Professor Joachim and Schyns, Professor Philippe and Giordano, Dr Bruno and Panzeri, Professor Stefano and Ince, Dr Robin
Authors: Giordano, B. L., Ince, R. A.A., Gross, J., Schyns, P. G., Panzeri, S., and Kayser, C.
College/School:College of Medical Veterinary and Life Sciences > Institute of Neuroscience and Psychology
Journal Name:eLife
Publisher:eLife Sciences Publications, Ltd
ISSN:2050-084X
ISSN (Online):2050-084X
Copyright Holders:Copyright © 2017 Giordano et al.
First Published:First published in eLife 6: e24763
Publisher Policy:Reproduced under a Creative Commons license

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
658701Pathways and mechanisms underlying the visual enhancement of hearing in challenging environments.Christoph KayserBiotechnology and Biological Sciences Research Council (BBSRC)BB/L027534/1INP - CENTRE FOR COGNITIVE NEUROIMAGING
670551DynaSensChristoph KayserEuropean Research Council (ERC)646657INP - CENTRE FOR COGNITIVE NEUROIMAGING
649631The neural representation of vocal emotion: representational similarity analysis and information-theoretic approachesJoachim GrossBiotechnology and Biological Sciences Research Council (BBSRC)BB/M009742/1INP - CENTRE FOR COGNITIVE NEUROIMAGING
597051Natural and modulated neural communication: State-dependent decoding and driving of human Brain Oscillations.Joachim GrossWellcome Trust (WELLCOTR)098433/Z/12/ZINP - CENTRE FOR COGNITIVE NEUROIMAGING
698281Brain Algorithmics: Reverse Engineering Dynamic Information Processing Networks from MEG time seriesPhilippe SchynsWellcome Trust (WELLCOTR)107802/Z/15/ZINP - CENTRE FOR COGNITIVE NEUROIMAGING