Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)

Filippone, M. and Engler, R. (2015) Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE). Journal of Machine Learning Research: Workshop and Conference Proceedings, 37, pp. 1015-1024.

Abstract

In applications of Gaussian processes where quantification of uncertainty is of primary interest, it is necessary to accurately characterize the posterior distribution over covariance parameters. This paper proposes an adaptation of the Stochastic Gradient Langevin Dynamics algorithm to draw samples from the posterior distribution over covariance parameters with negligible bias and without the need to compute the marginal likelihood. In Gaussian process regression, this has the enormous advantage that stochastic gradients can be computed by solving linear systems only. A novel unbiased linear systems solver based on parallelizable covariance matrix-vector products is developed to accelerate the unbiased estimation of gradients. The results demonstrate the possibility to enable scalable and exact (in a Monte Carlo sense) quantification of uncertainty in Gaussian processes without imposing any special structure on the covariance or reducing the number of input vectors.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Filippone, Dr Maurizio
• Authors: Filippone, M., and Engler, R.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Journal of Machine Learning Research: Workshop and Conference Proceedings
• Publisher: Microtome Publishing
• ISSN: 1938-7228
• Copyright Holders: Copyright © 2015 The Authors
• First Published: First published in Journal of Machine Learning Research (JMLR) Workshop and Conference Proceedings 37:1015-1024
• Publisher Policy: Reproduced with the permission of the authors.
• Data DOI: 10.5525/gla.researchdata.279

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
63329	1	Computational inference in systems biology	Dirk Husmeier	Engineering & Physical Sciences Research Council (EPSRC)	EP/L020319/1	M&S - STATISTICS