Stream-Based Recommendations: Online and Offline Evaluation as a Service

Kille, B., Lommatzsch, A., Turrin, R., Sereny, A., Larson, M., Brodt, T., Seiler, J. and Hopfgartner, F. (2015) Stream-Based Recommendations: Online and Offline Evaluation as a Service. In: 6th International Conference of the CLEF Initiative, Toulouse, France, 8-11 Sep 2015, pp. 487-507. ISBN 9783319240268 (doi: 10.1007/978-3-319-24027-5 48)

Abstract

Providing high-quality news recommendations is a challenging task because the set of potentially relevant news items changes continuously, the relevance of news highly depends on the context, and there are tight time constraints for computing recommendations. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms online and offline. In this paper, we discuss the objectives and challenges of the NewsREEL lab. We motivate the metrics used for benchmarking the recommender algorithms and explain the challenge dataset. In addition, we introduce the evaluation framework that we have developed. The framework makes possible the reproducible evaluation of recommender algorithms for stream data, taking into account recommender precision as well as the technical complexity of the recommender algorithms.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Hopfgartner, Dr Frank
• Authors: Kille, B., Lommatzsch, A., Turrin, R., Sereny, A., Larson, M., Brodt, T., Seiler, J., and Hopfgartner, F.
• College/School: College of Arts & Humanities > School of Humanities > Information Studies
• ISSN: 0302-9743
• ISBN: 9783319240268
• Copyright Holders: Copyright © 2015 Springer International Publishing
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher

Related URLs

• Organisation