Discovering Interpretable Representations for Both Deep Generative and Discriminative Models

Adel, T. , Ghahramani, Z. and Weller, A. (2018) Discovering Interpretable Representations for Both Deep Generative and Discriminative Models. In: 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 10-15 July 2018, pp. 50-59.

Abstract

Interpretability of representations in both deep generative and discriminative models is highly desirable. Current methods jointly optimize an objective combining accuracy and interpretability. However, this may reduce accuracy, and is not applicable to already trained models. We propose two interpretability frameworks. First, we provide an interpretable lens for an existing model. We use a generative model which takes as input the representation in an existing (generative or discriminative) model, weakly supervised by limited side information. Applying a flexible and invertible transformation to the input leads to an interpretable representation with no loss in accuracy. We extend the approach using an active learning strategy to choose the most useful side information to obtain, allowing a human to guide what "interpretable" means. Our second framework relies on joint optimization for a representation which is both maximally informative about the side information and maximally compressive about the non-interpretable data factors. This leads to a novel perspective on the relationship between compression and regularization. We also propose a new interpretability evaluation metric based on our framework. Empirically, we achieve state-of-the-art results on three datasets using the two proposed algorithms.

• Item Type: Conference Proceedings
• Additional Information: AW acknowledges support from the David MacKay Newton research fellowship at Darwin College and The Alan Turing Institute under EPSRC grant EP/N510129/1 and TU/B/000074.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Hesham, Dr Tameem Adel
• Authors: Adel, T., Ghahramani, Z., and Weller, A.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 2640-3498