Collapsed Variational Inference for Sum-Product Networks

Zhao, H., Adel, T. , Gordon, G. and Amos, B. (2016) Collapsed Variational Inference for Sum-Product Networks. In: 33rd International Conference on Machine Learning, ICML 2016, New York, USA, 19-24 June 2016, pp. 1310-1318.

Abstract

Sum-Product Networks (SPNs) are probabilistic inference machines that admit exact inference in linear time in the size of the network. Existing parameter learning approaches for SPNs are largely based on the maximum likelihood principle and are subject to overfitting compared to more Bayesian approaches. Exact Bayesian posterior inference for SPNs is computationally intractable. Even approximation techniques such as standard variational inference and posterior sampling for SPNs are computationally infeasible even for networks of moderate size due to the large number of local latent variables per instance. In this work, we propose a novel deterministic collapsed variational inference algorithm for SPNs that is computationally efficient, easy to implement and at the same time allows us to incorporate prior information into the optimization formulation. Extensive experiments show a significant improvement in accuracy compared with a maximum likelihood based approach.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Hesham, Dr Tameem Adel
• Authors: Zhao, H., Adel, T., Gordon, G., and Amos, B.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 2640-3498