TREC Incident Streams: Finding Actionable Information on Social Media

Mccreadie, R. , Buntain, C. and Soboroff, I. (2019) TREC Incident Streams: Finding Actionable Information on Social Media. In: 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2019), Valencia, Spain, 19-22 May 2019, pp. 691-705. ISBN 9788409104987

183409.pdf - Published Version
Available under License Creative Commons Attribution.


Publisher's URL:


The Text Retrieval Conference (TREC) Incident Streams track is a new initiative that aims to mature social media-based emergency response technology. This initiative advances the state of the art in this area through an evaluation challenge, which attracts researchers and developers from across the globe. The 2018 edition of the track provides a standardized evaluation methodology, an ontology of emergency-relevant social media information types, proposes a scale for information criticality, and releases a dataset containing fifteen test events and approximately 20,000 labeled tweets. Analysis of this dataset reveals a significant amount of actionable information on social media during emergencies (> 10%). While this data is valuable for emergency response efforts, analysis of the 39 state-of-the-art systems demonstrate a performance gap in identifying this data. We therefore find the current state-of-the-art is insufficient for emergency responders’ requirements, particularly for rare actionable information for which there is little prior training data available.

Item Type:Conference Proceedings
Keywords:Emergency management, crisis informatics, real-time, Twitter, categorization.
Glasgow Author(s) Enlighten ID:Mccreadie, Dr Richard
Authors: Mccreadie, R., Buntain, C., and Soboroff, I.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering > School of Computing Science
Research Group:Information, Data, Analysis
Copyright Holders:Copyright © 2019 The Authors
First Published:First published in Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management, Valencia (Spain), 19-22 May 2019, 691-705
Publisher Policy:Reproduced under a Creative Commons License
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record