Kay, J. W. and Ince, R. A.A. (2018) Exact partial information decompositions for Gaussian systems based on dependency constraints. Entropy, 20(4), 240. (doi: 10.3390/e20040240)
|
Text
159920.pdf - Published Version Available under License Creative Commons Attribution. 1MB |
Abstract
The Partial Information Decomposition, introduced by Williams P. L. et al. (2010), provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed by James R. G. et al. (2017) for computing a two-predictor partial information decomposition over discrete spaces. A lattice of maximum entropy probability models is constructed based on marginal dependency constraints, and the unique information that a particular predictor has about the target is defined as the minimum increase in joint predictor-target mutual information when that particular predictor-target marginal dependency is constrained. Here, we apply the Idep approach to Gaussian systems, for which the marginally constrained maximum entropy models are Gaussian graphical models. Closed form solutions for the Idep PID are derived for both univariate and multivariate Gaussian systems. Numerical and graphical illustrations are provided, together with practical and theoretical comparisons of the Idep PID with the minimum mutual information partial information decomposition (Immi), which was discussed by Barrett A. B. (2015). The results obtained using Idep appear to be more intuitive than those given with other methods, such as Immi, in which the redundant and unique information components are constrained to depend only on the predictor-target marginal distributions. In particular, it is proved that the Immi method generally produces larger estimates of redundancy and synergy than does the Idep method. In discussion of the practical examples, the PIDs are complemented by the use of tests of deviance for the comparison of Gaussian graphical models.
Item Type: | Articles |
---|---|
Status: | Published |
Refereed: | Yes |
Glasgow Author(s) Enlighten ID: | Ince, Dr Robin and Kay, Dr James |
Authors: | Kay, J. W., and Ince, R. A.A. |
College/School: | College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience College of Science and Engineering > School of Mathematics and Statistics > Statistics |
Journal Name: | Entropy |
Publisher: | MDPI |
ISSN: | 1099-4300 |
ISSN (Online): | 1099-4300 |
Published Online: | 30 March 2018 |
Copyright Holders: | Copyright © 2018 The Authors |
First Published: | First published in Entropy 20(4): 240 |
Publisher Policy: | Reproduced under a Creative Commons License |
University Staff: Request a correction | Enlighten Editors: Update this record