Bolam, J., Boyle, S. C. , Ince, R. A.A. and Delis, I. (2022) Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions. NeuroImage, 247, 118841. (doi: 10.1016/j.neuroimage.2021.118841) (PMID:34952232)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
261342.pdf - Published Version Available under License Creative Commons Attribution. 2MB |
Abstract
When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent ‘early’ sensory processing benefits, or ‘late’ post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate linear discriminant analysis to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an ‘Early’ component (∼100ms-110ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli), and a ‘Late’ component (∼330ms-340ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed hierarchical drift diffusion model, revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of ‘early’ sensory processing, as well as ‘late’ post-sensory neural representations of associative congruency to perceptual decision formation.
| Item Type: | Articles |
|---|---|
| Additional Information: | This work was supported by the European Commission (H2020-MSCA-IF-2018/845884, “NeuCoDe” to I.D.) and the Physiological Society (2018 Research Grant Scheme to I.D.). S.C.B was funded by the Biotechnology and Biological Sciences Research Council (BBSRC) through a BBSRC DTP Studentship (Grant Number BB/L027534/1). |
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Boyle, Miss Stephanie and Delis, Dr Ioannis and Ince, Dr Robin |
| Creator Roles: | Boyle, S. C.Data curation, Funding acquisition, Investigation, Methodology Ince, R. A.A.Data curation, Investigation, Methodology, Validation Delis, I.Conceptualization, Funding acquisition, Resources, Supervision, Validation, Writing – original draft, Writing – review and editing |
| Authors: | Bolam, J., Boyle, S. C., Ince, R. A.A., and Delis, I. |
| College/School: | College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience |
| Journal Name: | NeuroImage |
| Publisher: | Elsevier |
| ISSN: | 1053-8119 |
| ISSN (Online): | 1095-9572 |
| Published Online: | 21 December 2021 |
| Copyright Holders: | Copyright © 2022 The Authors |
| First Published: | First published in NeuroImage 247: 118841 |
| Publisher Policy: | Reproduced under a Creative Commons License |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information