Emulation-accelerated Hamiltonian Monte Carlo algorithms for parameter estimation and uncertainty quantification in differential equation models

Paun, L. M. and Husmeier, D. (2022) Emulation-accelerated Hamiltonian Monte Carlo algorithms for parameter estimation and uncertainty quantification in differential equation models. Statistics and Computing, 32(1), 1. (doi: 10.1007/s11222-021-10060-4)

Abstract

We propose to accelerate Hamiltonian and Lagrangian Monte Carlo algorithms by coupling them with Gaussian processes for emulation of the log unnormalised posterior distribution. We provide proofs of detailed balance with respect to the exact posterior distribution for these algorithms, and validate the correctness of the samplers’ implementation by Geweke consistency tests. We implement these algorithms in a delayed acceptance (DA) framework, and investigate whether the DA scheme can offer computational gains over the standard algorithms. A comparative evaluation study is carried out to assess the performance of the methods on a series of models described by differential equations, including a real-world application of a 1D fluid-dynamics model of the pulmonary blood circulation. The aim is to identify the algorithm which gives the best trade-off between accuracy and computational efficiency, to be used in nonlinear DE models, which are computationally onerous due to repeated numerical integrations in a Bayesian analysis. Results showed no advantage of the DA scheme over the standard algorithms with respect to several efficiency measures based on the effective sample size for most methods and DE models considered. These gradient-driven algorithms register a high acceptance rate, thus the number of expensive forward model evaluations is not significantly reduced by the first emulator-based stage of DA. Additionally, the Lagrangian Dynamical Monte Carlo and Riemann Manifold Hamiltonian Monte Carlo tended to register the highest efficiency (in terms of effective sample size normalised by the number of forward model evaluations), followed by the Hamiltonian Monte Carlo, and the No U-turn sampler tended to be the least efficient.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Paun, Dr Mihaela and Husmeier, Professor Dirk
• Authors: Paun, L. M., and Husmeier, D.
• College/School: College of Science and Engineering > School of Mathematics and Statistics > Statistics
• Journal Name: Statistics and Computing
• Publisher: Springer
• ISSN: 0960-3174
• ISSN (Online): 1573-1375
• Published Online: 23 November 2021
• Copyright Holders: Copyright © 2021 The Authors
• First Published: First published in Statistics and Computing 32(1): 1
• Publisher Policy: Reproduced under a Creative Commons License

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
172141		EPSRC Centre for Multiscale soft tissue mechanics with application to heart & cancer	Raymond Ogden	Engineering and Physical Sciences Research Council (EPSRC)	EP/N014642/1	M&S - Mathematics
308255		The SofTMech Statistical Emulation and Translation Hub	Dirk Husmeier	Engineering and Physical Sciences Research Council (EPSRC)	EP/T017899/1	M&S - Statistics