Scalable distributed event detection for Twitter

McCreadie, R., Macdonald, C., Ounis, I. , Osborne, M. and Petrovic, S. (2013) Scalable distributed event detection for Twitter. In: 2013 IEEE International Conference on Big Data, Santa Clara, CA, USA, 6-9 Oct 2013, (doi:10.1109/BigData.2013.6691620)

[img]
Preview
Text
89118.pdf - Accepted Version

520kB

Publisher's URL: http://dx.doi.org/10.1109/BigData.2013.6691620

Abstract

Social media streams, such as Twitter, have shown themselves to be useful sources of real-time information about what is happening in the world. Automatic detection and tracking of events identified in these streams have a variety of real-world applications, e.g. identifying and automatically reporting road accidents for emergency services. However, to be useful, events need to be identified within the stream with a very low latency. This is challenging due to the high volume of posts within these social streams. In this paper, we propose a novel event detection approach that can both effectively detect events within social streams like Twitter and can scale to thousands of posts every second. Through experimentation on a large Twitter dataset, we show that our approach can process the equivalent to the full Twitter Firehose stream, while maintaining event detection accuracy and outperforming an alternative distributed event detection system.

Item Type:Conference Proceedings
Keywords:System analysis and design, event detection, distributed processing, large-scale systems, scalability, storm, Big Data
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Macdonald, Dr Craig and Ounis, Professor Iadh and Mccreadie, Mr Richard
Authors: McCreadie, R., Macdonald, C., Ounis, I., Osborne, M., and Petrovic, S.
College/School:College of Science and Engineering > School of Computing Science
Research Group:School of Computing Science - Terrier Team
Copyright Holders:Copyright © 2013 IEEE
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record