Adapting Learned Sparse Retrieval for Long Documents

Nguyen, T., MacAvaney, S. and Yates, A. (2023) Adapting Learned Sparse Retrieval for Long Documents. In: 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR23), Taipei, Taiwan, 23-27 July 2023, pp. 1781-1785. ISBN 9781450394086 (doi: 10.1145/3539618.3591943)

Abstract

Learned sparse retrieval (LSR) is a family of neural retrieval methods that transform queries and documents into sparse weight vectors aligned with a vocabulary. While LSR approaches like Splade work well for short passages, it is unclear how well they handle longer documents. We investigate existing aggregation approaches for adapting LSR to longer documents and find that proximal scoring is crucial for LSR to handle long documents. To leverage this property, we proposed two adaptations of the Sequential Dependence Model (SDM) to LSR: ExactSDM and SoftSDM. ExactSDM assumes only exact query term dependence, while SoftSDM uses potential functions that model the dependence of query terms and their expansion terms (i.e., terms identified using a transformer's masked language modeling head). Experiments on the MSMARCO Document and TREC Robust04 datasets demonstrate that both ExactSDM and SoftSDM outperform existing LSR aggregation approaches for different document length constraints. Surprisingly, SoftSDM does not provide any performance benefits over ExactSDM. This suggests that soft proximity matching is not necessary for modeling term dependence in LSR. Overall, this study provides insights into handling long documents with LSR, proposing adaptations that improve its performance.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: MacAvaney, Dr Sean
• Authors: Nguyen, T., MacAvaney, S., and Yates, A.
• College/School: College of Science and Engineering > School of Computing Science
• Research Centre: College of Science and Engineering > School of Computing Science > IDA Section > GPU Cluster
• ISBN: 9781450394086
• Copyright Holders: Copyright © 2023 The Authors
• First Published: First published in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR23), pp. 1781–1785
• Publisher Policy: Reproduced with the permission of the publisher

Related URLs

• Organisation