Variational Sparse Coding

Tonolini, F., Jensen, B. S. and Murray-Smith, R. (2019) Variational Sparse Coding. In: Conference on Uncertainty in Artificial Intelligence (UAI 2019), Tel Aviv, Israel, 22-25 July 2019,

191553.pdf - Accepted Version


Publisher's URL:


Unsupervised discovery of interpretable features and controllable generation with highdimensional data are currently major challenges in machine learning, with applications in data visualisation, clustering and artificial data synthesis. We propose a model based on variational auto-encoders (VAEs) in which interpretation is induced through latent space sparsity with a mixture of Spike and Slab distributions as prior. We derive an evidence lower bound for this model and propose a specific training method for recovering disentangled features as sparse elements in latent vectors. In our experiments, we demonstrate superior disentanglement performance to standard VAE approaches when an estimate of the number of true sources of variation is not available and objects display different combinations of attributes. Furthermore, the new model provides unique capabilities, such as recovering feature exploitation, synthesising samples that share attributes with a given input object and controlling both discrete and continuous features upon generation.

Item Type:Conference Proceedings
Glasgow Author(s) Enlighten ID:Murray-Smith, Professor Roderick and Tonolini, Francesco and Jensen, Dr Bjorn
Authors: Tonolini, F., Jensen, B. S., and Murray-Smith, R.
College/School:College of Science and Engineering
College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright © 2019 The Authors
Publisher Policy:Reproduced with the permission of the author
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
6672319UK Quantum Technology Hub in Enhanced Quantum ImagingMiles PadgettEngineering and Physical Sciences Research Council (EPSRC)EP/M01326X/1S&E P&A - PHYSICS & ASTRONOMY
3009820Exploiting Closed-Loop Aspects in Computationally and Data Intensive AnalyticsRoderick Murray-SmithEngineering and Physical Sciences Research Council (EPSRC)EP/R018634/1Computing Science