Reproducing Personalised Session Search over the AOL Query Log

MacAvaney, S. , Macdonald, C. and Ounis, I. (2022) Reproducing Personalised Session Search over the AOL Query Log. In: 44th European Conference on Information Retrieval (ECIR 2022), Stavanger, Norway, 10-14 Apr 2022, pp. 627-640. ISBN 9783030997359 (doi: 10.1007/978-3-030-99736-6_42)

Abstract

Despite its troubled past, the AOL Query Log continues to be an important resource to the research community—particularly for tasks like search personalisation. When using the query log these ranking experiments, little attention is usually paid to the document corpus. Recent work typically uses a corpus containing versions of the documents collected long after the log was produced. Given that web documents are prone to change over time, we study the differences present between a version of the corpus containing documents as they appeared in 2017 (which has been used by several recent works) and a new version we construct that includes documents close to as they appeared at the time the query log was produced (2006). We demonstrate that this new version of the corpus has a far higher coverage of documents present in the original log (93%) than the 2017 version (55%). Among the overlapping documents, the content often differs substantially. Given these differences, we re-conduct session search experiments that originally used the 2017 corpus and find that when using our corpus for training or evaluation, system performance improves. We place the results in context by introducing recent adhoc ranking baselines. We also confirm the navigational nature of the queries in the AOL corpus by showing that including the URL substantially improves performance across a variety of models. Our version of the corpus can be easily reconstructed by other researchers and is included in the ir-datasets package.

• Item Type: Conference Proceedings
• Additional Information: We acknowledge EPSRC grant EP/R018634/1: Closed-Loop Data Science for Complex, Computationally- & Data-Intensive Analytics.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: MacAvaney, Dr Sean and Macdonald, Professor Craig and Ounis, Professor Iadh
• Authors: MacAvaney, S., Macdonald, C., and Ounis, I.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 0302-9743
• ISBN: 9783030997359
• Published Online: 10 April 2022
• Copyright Holders: Copyright © 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
• First Published: First published in Advances in Information Retrieval ECIR 2022. Lecture Notes in Computer Science, vol 13185
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
300982		Exploiting Closed-Loop Aspects in Computationally and Data Intensive Analytics	Roderick Murray-Smith	Engineering and Physical Sciences Research Council (EPSRC)	EP/R018634/1	Computing Science