A query-basis approach to parametrizing novelty-biased cumulative gain

Leelanupab, T., Zuccon, G. and Jose, J.M. (2011) A query-basis approach to parametrizing novelty-biased cumulative gain. Lecture Notes in Computer Science, 6931, pp. 327-331. (doi: 10.1007/978-3-642-23318-0_32)

Abstract

Novelty-biased cumulative gain (α-NDCG) has become the de facto measure within the information retrieval (IR) community for evaluating retrieval systems in the context of sub-topic retrieval. Setting the incorrect value of parameter α in α-NDCG prevents the measure from behaving as desired in particular circumstances. In fact, when α is set according to common practice (i.e. α = 0.5), the measure favours systems that promote redundant relevant sub-topics rather than provide novel relevant ones. Recognising this characteristic of the measure is important because it affects the comparison and the ranking of retrieval systems. We propose an approach to overcome this problem by defining a safe threshold for the value of α on a query basis. Moreover, we study its impact on system rankings through a comprehensive simulation.

• Item Type: Articles
• Additional Information: Presented at ICTIR'11: Third International Conference on Advances in Information Retrieval Theory, The University Centre of Bertinoro, Italy, 12-14 September 2011.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Jose, Professor Joemon and Leelanupab, Mr Teerapong and Zuccon, Mr Guido
• Authors: Leelanupab, T., Zuccon, G., and Jose, J.M.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Lecture Notes in Computer Science
• Publisher: Springer
• ISSN: 0302-9743
• ISSN (Online): 0302-9743