Ekstrand-Abueg, M., McCreadie, R. , Pavlu, V. and Diaz, F. (2016) A Study of Realtime Summarization Metrics. In: 25th ACM International Conference on Information and Knowledge Management (CIKM 2016), Indianapolis, IN, USA, 24-28 Oct 2016, pp. 2125-2130. ISBN 9781450340731 (doi: 10.1145/2983323.2983653)
|
Text
148597.pdf - Accepted Version 833kB |
Abstract
Unexpected news events, such as natural disasters or other human tragedies, create a large volume of dynamic text data from official news media as well as less formal social media. Automatic real-time text summarization has become an important tool for quickly transforming this overabundance of text into clear, useful information for end-users including affected individuals, crisis responders, and interested third parties. Despite the importance of real-time summarization systems, their evaluation is not well understood as classic methods for text summarization are inappropriate for real-time and streaming conditions. The TREC 2013-2015 Temporal Summarization (TREC-TS) track was one of the first evaluation campaigns to tackle the challenges of real-time summarization evaluation, introducing new metrics, ground-truth generation methodology and dataset. In this paper, we present a study of TREC-TS track evaluation methodology, with the aim of documenting its design, analyzing its effectiveness, as well as identifying improvements and best practices for the evaluation of temporal summarization systems.
| Item Type: | Conference Proceedings |
|---|---|
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Mccreadie, Dr Richard |
| Authors: | Ekstrand-Abueg, M., McCreadie, R., Pavlu, V., and Diaz, F. |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| College/School: | College of Science and Engineering > School of Computing Science |
| Research Group: | Information Retrieval |
| ISBN: | 9781450340731 |
| Copyright Holders: | Copyright © 2016 The Authors |
| First Published: | First published in CIKM '16 Proceedings of the 25th ACM International on Conference on Information and Knowledge Management: 2125-2130 |
| Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
University Staff: Request a correction | Enlighten Editors: Update this record
Deposit and Record Details
Deposit and Record Details