Reproducing Personalised Session Search over the AOL Query Log

MacAvaney, S. , Macdonald, C. and Ounis, I. (2022) Reproducing Personalised Session Search over the AOL Query Log. In: 44th European Conference on Information Retrieval (ECIR 2022), Stavanger, Norway, 10-14 Apr 2022, pp. 627-640. ISBN 9783030997359 (doi: 10.1007/978-3-030-99736-6_42)

[img] Text
259616.pdf - Accepted Version



Despite its troubled past, the AOL Query Log continues to be an important resource to the research community—particularly for tasks like search personalisation. When using the query log these ranking experiments, little attention is usually paid to the document corpus. Recent work typically uses a corpus containing versions of the documents collected long after the log was produced. Given that web documents are prone to change over time, we study the differences present between a version of the corpus containing documents as they appeared in 2017 (which has been used by several recent works) and a new version we construct that includes documents close to as they appeared at the time the query log was produced (2006). We demonstrate that this new version of the corpus has a far higher coverage of documents present in the original log (93%) than the 2017 version (55%). Among the overlapping documents, the content often differs substantially. Given these differences, we re-conduct session search experiments that originally used the 2017 corpus and find that when using our corpus for training or evaluation, system performance improves. We place the results in context by introducing recent adhoc ranking baselines. We also confirm the navigational nature of the queries in the AOL corpus by showing that including the URL substantially improves performance across a variety of models. Our version of the corpus can be easily reconstructed by other researchers and is included in the ir-datasets package.

Item Type:Conference Proceedings
Additional Information:We acknowledge EPSRC grant EP/R018634/1: Closed-Loop Data Science for Complex, Computationally- & Data-Intensive Analytics.
Glasgow Author(s) Enlighten ID:MacAvaney, Dr Sean and Macdonald, Professor Craig and Ounis, Professor Iadh
Authors: MacAvaney, S., Macdonald, C., and Ounis, I.
College/School:College of Science and Engineering > School of Computing Science
Published Online:10 April 2022
Copyright Holders:Copyright © 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
First Published:First published in Advances in Information Retrieval ECIR 2022. Lecture Notes in Computer Science, vol 13185
Publisher Policy:Reproduced in accordance with the publisher copyright policy
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
300982Exploiting Closed-Loop Aspects in Computationally and Data Intensive AnalyticsRoderick Murray-SmithEngineering and Physical Sciences Research Council (EPSRC)EP/R018634/1Computing Science