Extrinsic Bayesian optimization on manifolds

Fang, Y., Niu, M., Cheung, P. and Lin, L. (2023) Extrinsic Bayesian optimization on manifolds. Algorithms, 16(2), 117. (doi: 10.3390/a16020117)

[img] Text
292774.pdf - Published Version
Available under License Creative Commons Attribution.

765kB

Abstract

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.

Item Type:Articles
Additional Information:The authors acknowledge the generous support of NSF grants DMS CAREER 1654579 and DMS 2113642. M. N acknowledges support for this paper from EPSRC grants EP/W021595/1 and EP/X5257161/1.
Keywords:Bayesian optimization, optimizations on manifolds, embedding, extrinsic gaussian process.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Niu, Dr Mu
Creator Roles:
Niu, M.Conceptualization, Methodology, Software, Writing – review and editing
Authors: Fang, Y., Niu, M., Cheung, P., and Lin, L.
College/School:College of Science and Engineering > School of Mathematics and Statistics > Statistics
Journal Name:Algorithms
Publisher:MDPI
ISSN:1999-4893
ISSN (Online):1999-4893
Published Online:15 February 2023
Copyright Holders:Copyright © 2023 The Authors
First Published:First published in Algorithms 16(2): 117
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record