CrisisViT: A Robust Vision Transformer for Crisis Image Classification

Long, Z. , Mccreadie, R. and Imran, M. (2023) CrisisViT: A Robust Vision Transformer for Crisis Image Classification. In: 20th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2023), Omaha, NE, USA, 28-31 May 2023, pp. 309-319. ISBN 9798218217495 (doi: 10.59297/SDSM9194)

[img] Text
295808.pdf - Accepted Version

328kB

Abstract

In times of emergency, crisis response agencies need to quickly and accurately assess the situation on the ground in order to deploy relevant services and resources. However, authorities often have to make decisions based on limited information, as data on affected regions can be scarce until local response services can provide first-hand reports. Fortunately, the widespread availability of smartphones with high-quality cameras has made citizen journalism through social media a valuable source of information for crisis responders. However, analyzing the large volume of images posted by citizens requires more time and effort than is typically available. To address this issue, this paper proposes the use of state-of-the-art deep neural models for automatic image classification/tagging, specifically by adapting transformer-based architectures for crisis image classification (CrisisViT). We leverage the new Incidents1M crisis image dataset to develop a range of new transformer-based image classification models. Through experimentation over the standard Crisis image benchmark dataset, we demonstrate that the CrisisViT models significantly outperform previous approaches in emergency type, image relevance, humanitarian category, and damage severity classification. Additionally, we show that the new Incidents1M dataset can further augment the CrisisViT models resulting in an additional 1.25% absolute accuracy gain.

Item Type:Conference Proceedings
Keywords:Social media classification, crisis management, deep learning, vision transformers, supervised learning.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Mccreadie, Dr Richard and LONG, ZIJUN
Authors: Long, Z., Mccreadie, R., and Imran, M.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering
College of Science and Engineering > School of Computing Science
Research Centre:College of Science and Engineering > School of Computing Science > IDA Section > GPU Cluster
Research Group:Information Retrieval
ISSN:2411-3387
ISBN:9798218217495
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record