Quantitatively Comparing Predictive Models with the Partial Information Decomposition

Daube, C., Giordano, B., Schyns, P. G. and Ince, R. A.A. (2019) Quantitatively Comparing Predictive Models with the Partial Information Decomposition. 2019 Conference on Cognitive Computational Neuroscience, Berlin, Germany, 13-16 Sep 2019. pp. 838-840. (doi: 10.32470/CCN.2019.1142-0)

Abstract

There is increasing focus in cognitive and computational neuroscience on the use of encoding and decoding models to gain insight into cognitive processing. Frequently, encoding models are fit to a number of different features sets, and the out-of-sample predictive performance of the resulting models is compared. However, to gain the maximum benefit from this modelling, we need to go beyond simply ranking model performance in terms of absolute predictive power. We also need to directly compare and relate the predictions between models, to gain insight into which models are predicting common vs unique aspects of the neural response. The Partial Information Decomposition (PID) provides a principled theoretical framework to address this question, as it decomposes the total predictive performance of two models into redundant (overlapping), unique, and synergistic parts. We show that like classical information theoretic quantities, variance decomposition approaches conflate synergy and redundancy and so could provide a misleading view of the unique predictive power of a model. We also suggest how the use of encoding models and PID can help interpret decoding models.

• Item Type: Conference or Workshop Item
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Daube, Dr Christoph and Schyns, Professor Philippe and Giordano, Dr Bruno and Ince, Dr Robin
• Authors: Daube, C., Giordano, B., Schyns, P. G., and Ince, R. A.A.
• College/School: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Copyright Holders: Copyright © 2019 The Authors
• First Published: First published in 2019 Confernece on Cognitive Computational Neuroscience
• Publisher Policy: Reproduced under a Creative Commons License

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
304240		Beyond Pairwise Connectivity: developing an information theoretic hypergraph methodology for multi-modal resting state neuroimaging analysis	Robin Ince	Wellcome Trust (WELLCOTR)	214120/Z/18/Z	Centre for Cognitive Neuroimaging
172413		Brain Algorithmics: Reverse Engineering Dynamic Information Processing Networks from MEG time series	Philippe Schyns	Wellcome Trust (WELLCOTR)	107802/Z/15/Z	Centre for Cognitive Neuroimaging
172046		Visual Commonsense for Scene Understanding	Philippe Schyns	Engineering and Physical Sciences Research Council (EPSRC)	EP/N019261/1	Centre for Cognitive Neuroimaging