Overview of CLEF NEWSREEL 2014: News Recommendations Evaluation Labs

Kille, B., Brodt, T., Heintz, T., Hopfgartner, F., Lommatzsch, A. and Seiler, J. (2014) Overview of CLEF NEWSREEL 2014: News Recommendations Evaluation Labs. Working Paper. CEUR.

[img]
Preview
Text
101285.pdf - Published Version

779kB

Publisher's URL: http://ceur-ws.org/Vol-1180/CLEF2014wn-Newsreel-Kille2014.pdf

Abstract

This paper summarises objectives, organisation, and results of the first news recommendation evaluation lab (NEWSREEL 2014). NEWSREEL targeted the evaluation of news recommendation algorithms in the form of a campaignstyle evaluation lab. Participants had the chance to apply two types of evaluation schemes. On the one hand, participants could apply their algorithms onto a data set. We refer to this setting as off-line evaluation. On the other hand, participants could deploy their algorithms on a server to interactively receive recommendation requests. We refer to this setting as on-line evaluation. This setting ought to reveal the actual performance of recommendation methods. The competition strived to illustrate differences between evaluation with historical data and actual users. The on-line evaluation does reflect all requirements which active recommender systems face in practise. These requirements include real-time responses and large-scale data volumes. We present the competition’s results and discuss commonalities regarding participants’ approaches.

Item Type:Research Reports or Papers (Working Paper)
Status:Published
Glasgow Author(s) Enlighten ID:Hopfgartner, Dr Frank
Authors: Kille, B., Brodt, T., Heintz, T., Hopfgartner, F., Lommatzsch, A., and Seiler, J.
College/School:College of Arts > School of Humanities > Humanities Advanced Technology and Information Institute (HATII)
Publisher:CEUR
Copyright Holders:Copyright © 2014 CEUR
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher

University Staff: Request a correction | Enlighten Editors: Update this record