Incident Streams 2020: TREC-IS in the Time of COVID-19

Buntain, C., Mccreadie, R. and Soboroff, I. (2021) Incident Streams 2020: TREC-IS in the Time of COVID-19. In: 18th International Conference on Information Systems for Crisis Response and Management, Blacksburg, VA, USA, 23-26 May 2021, pp. 621-639.

Abstract

Between 2018 and 2019, the Incident Streams track (TREC-IS) has developed standard approaches for classifying the types and criticality of information shared in online social spaces during crises, but the introduction of SARS-CoV-2 has shifted the landscape of online crises substantially. While prior editions of TREC-IS have lacked data on large-scale public-health emergencies as these events are exceedingly rare, COVID-19 has introduced an over-abundance of potential data, and significant open questions remain about how existing approaches to crisis informatics and datasets built on other emergencies adapt to this new context. This paper describes how the 2020 edition of TREC-IS has addressed these dual issues by introducing a new COVID-19-specific task for evaluating generalization of existing COVID-19 annotation and system performance to this new context, applied to 11 regions across the globe. TREC-IS has also continued expanding its set of target crises, adding 29 new events and expanding the collection of event types to include explosions, fires, and general storms, making for a total of 9 event types in addition to the new COVID-19 events. Across these events, TREC-IS has made available 478,110 COVID-related messages and 282,444 crisis-related messages for participant systems to analyze, of which 14,835 COVID-related and 19,784 crisis-related messages have been manually annotated. Analyses of these new datasets and participant systems demonstrate first that both the distributions of information type and priority of information vary between general crises and COVID-19-related discussion. Secondly, despite these differences, results suggest leveraging general crisis data in the COVID-19 context improves performance over baselines. Using these results, we provide guidance on which information types appear most consistent between general crises and COVID-19.

• Item Type: Conference Proceedings
• Keywords: Emergency management, crisis informatics, Twitter, categorization, prioritization, COVID-19.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Mccreadie, Dr Richard
• Authors: Buntain, C., Mccreadie, R., and Soboroff, I.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• College/School: College of Science and Engineering > School of Computing Science
• Research Group: Information Retrieval
• ISSN: 9781949373615
• Published Online: 23 May 2021

Related URLs

• Publisher