Mccreadie, R. , Buntain, C. and Soboroff, I. (2019) TREC Incident Streams: Finding Actionable Information on Social Media. In: 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2019), Valencia, Spain, 19-22 May 2019, pp. 691-705. ISBN 9788409104987
|
Text
183409.pdf - Published Version Available under License Creative Commons Attribution. 700kB |
Publisher's URL: http://idl.iscram.org/files/richardmccreadie/2019/1867_RichardMcCreadie_etal2019.pdf
Abstract
The Text Retrieval Conference (TREC) Incident Streams track is a new initiative that aims to mature social media-based emergency response technology. This initiative advances the state of the art in this area through an evaluation challenge, which attracts researchers and developers from across the globe. The 2018 edition of the track provides a standardized evaluation methodology, an ontology of emergency-relevant social media information types, proposes a scale for information criticality, and releases a dataset containing fifteen test events and approximately 20,000 labeled tweets. Analysis of this dataset reveals a significant amount of actionable information on social media during emergencies (> 10%). While this data is valuable for emergency response efforts, analysis of the 39 state-of-the-art systems demonstrate a performance gap in identifying this data. We therefore find the current state-of-the-art is insufficient for emergency responders’ requirements, particularly for rare actionable information for which there is little prior training data available.
Item Type: | Conference Proceedings |
---|---|
Keywords: | Emergency management, crisis informatics, real-time, Twitter, categorization. |
Status: | Published |
Refereed: | Yes |
Glasgow Author(s) Enlighten ID: | Mccreadie, Dr Richard |
Authors: | Mccreadie, R., Buntain, C., and Soboroff, I. |
Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
College/School: | College of Science and Engineering > School of Computing Science |
Research Group: | Information, Data, Analysis |
ISSN: | 2411-3387 |
ISBN: | 9788409104987 |
Copyright Holders: | Copyright © 2019 The Authors |
First Published: | First published in Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management, Valencia (Spain), 19-22 May 2019, 691-705 |
Publisher Policy: | Reproduced under a Creative Commons License |
Related URLs: |
University Staff: Request a correction | Enlighten Editors: Update this record