Variational Bayesian multinomial probit regression with Gaussian process priors

Girolami, M. and Rogers, S. (2006) Variational Bayesian multinomial probit regression with Gaussian process priors. Neural Computation, 18(8), pp. 1790-1817.

[img]
Preview
Text
Rogers2_MIT.pdf

473kB

Publisher's URL: http://neco.mitpress.org/

Abstract

It is well known in the statistics literature that augmenting binary and polychotomous response models with Gaussian latent variables enables exact Bayesian analysis via Gibbs sampling from the parameter posterior. By adopting such a data augmentation strategy, dispensing with priors over regression coefficients in favour of Gaussian Process (GP) priors over functions, and employing variational approximations to the full posterior we obtain efficient computational methods for Gaussian Process classification in the multi-class setting. The model augmentation with additional latent variables ensures full a posteriori class coupling whilst retaining the simple a priori independent GP covariance structure from which sparse approximations, such as multi-class Informative Vector Machines (IVM), emerge in a very natural and straightforward manner. This is the first time that a fully Variational Bayesian treatment for multi-class GP classification has been developed without having to resort to additional explicit approximations to the non-Gaussian likelihood term. Empirical comparisons with exact analysis via MCMC and Laplace approximations illustrate the utility of the variational approximation as a computationally economic alternative to full MCMC and it is shown to be more accurate than the Laplace approximation.

Item Type:Articles
Additional Information:Probit regression; gaussian process, variational bayes, multi-class classification
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Rogers, Dr Simon and Girolami, Prof Mark
Authors: Girolami, M., and Rogers, S.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Neural Computation
Publisher:MIT Press
ISSN:0899-7667
Copyright Holders:Copyright © 2006 MIT Press
First Published:First published in Neural Computation 18(8):1790-1817
Publisher Policy:Reproduced with permission of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record