Explaining holistic image regressors and classifiers in urban analytics with plausible counterfactuals

Law, S., Hasegawa, R., Paige, B., Russell, C. and Elliott, A. (2023) Explaining holistic image regressors and classifiers in urban analytics with plausible counterfactuals. International Journal of Geographical Information Science, 37, pp. 2575-2596. (doi: 10.1080/13658816.2023.2214592)

Abstract

We propose a new form of plausible counterfactual explanation designed to explain the behaviour of computer vision systems used in urban analytics that make predictions based on properties across the entire image, rather than specific regions of it. We illustrate the merits of our approach by explaining computer vision models used to analyse street imagery, which are now widely used in GeoAI and urban analytics. Such explanations are important in urban analytics as researchers and practioners are increasingly reliant on it for decision making. Finally, we perform a user study that demonstrate our approach can be used by non-expert users, who might not be machine learning experts, to be more confident and to better understand the behaviour of image-based classifiers/regressors for street view analysis. Furthermore, the method can potentially be used as an engagement tool to visualise how public spaces can plausibly look like. The limited realism of the counterfactuals is a concern which we hope to improve in the future.

• Item Type: Articles
• Additional Information: This project was supported by The Alan Turing Institute Turing Fellows Small Projects Funds. CR contributed to this work as part of his duties in the Trustworthy Auditing for AI project at the Oxford Internet Institute.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Elliott, Dr Andrew
• Authors: Law, S., Hasegawa, R., Paige, B., Russell, C., and Elliott, A.
• College/School: College of Science and Engineering > School of Mathematics and Statistics > Statistics
• Journal Name: International Journal of Geographical Information Science
• Publisher: Taylor & Francis
• ISSN: 1365-8816
• ISSN (Online): 1365-8824
• Published Online: 23 May 2023
• Copyright Holders: Copyright © 2023 The Author(s)
• First Published: First published in International Journal of Geographical Information Science 37:2575-2596
• Publisher Policy: Reproduced under a Creative Commons licence