Efficiently estimating retrievability bias

Wilkie, C. and Azzopardi, L. (2014) Efficiently estimating retrievability bias. Lecture Notes in Computer Science, 8416, pp. 720-726. (doi: 10.1007/978-3-319-06028-6_82)

Abstract

Retrievability is the measure of how easily a document can be retrieved using a particular retrieval system. The extent to which a retrieval system favours certain documents over others (as expressed by their retrievability scores) determines the level of bias the system imposes on a collection. Recently it has been shown that it is possible to tune a retrieval system by minimising the retrievability bias. However, to perform such a retrievability analysis often requires posing millions upon millions of queries. In this paper, we examine how many queries are needed to obtain a reliable and useful approximation of the retrievability bias imposed by the system, and an estimate of the individual retrievability of documents in the collection. We find that a reliable estimate of retrievability bias can be obtained, in some cases, with 90% less queries than are typically used while estimating document retrievability can be done with up to 60% less queries.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Azzopardi, Dr Leif and Wilkie, Mr Colin
• Authors: Wilkie, C., and Azzopardi, L.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Lecture Notes in Computer Science
• Publisher: Springer
• ISSN: 0302-9743
• ISSN (Online): 0302-9743