PARADE: passage representation aggregation for document reranking

Li, C., Yates, A. , MacAvaney, S. , He, B. and Sun, Y. (2023) PARADE: passage representation aggregation for document reranking. ACM Transactions on Information Systems, (doi: 10.1145/3600088) (Early Online Publication)

Abstract

Pre-trained transformer models, such as BERT and T5, have shown to be highly effective at ad-hoc passage and document ranking. Due to the inherent sequence length limits of these models, they need to process document passages one at a time rather than processing the entire document sequence at once. Although several approaches for aggregating passage-level signals into a document-level relevance score have been proposed, there has yet to be an extensive comparison of these techniques. In this work, we explore strategies for aggregating relevance signals from a document’s passages into a final ranking score. We find that passage representation aggregation techniques can significantly improve over score aggregation techniques proposed in prior work, such as taking the maximum passage score. We call this new approach PARADE. In particular, PARADE can significantly improve results on collections with broad information needs where relevance signals can be spread throughout the document (such as TREC Robust04 and GOV2). Meanwhile, less complex aggregation techniques may work better on collections with an information need that can often be pinpointed to a single passage (such as TREC DL and TREC Genomics). We also conduct efficiency analyses and highlight several strategies for improving transformer-based aggregation.

• Item Type: Articles
• Additional Information: This work was supported in part by the National Natural Science Foundation of China (Grant No. 62272439), Google Cloud, and the Google TPU Research Cloud (TRC).
• Status: Early Online Publication
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: MacAvaney, Dr Sean and Yates, Professor Andrew and He, Mr Ben
• Authors: Li, C., Yates, A., MacAvaney, S., He, B., and Sun, Y.
• College/School: College of Medical Veterinary and Life Sciences > School of Infection & Immunity; College of Science and Engineering > School of Computing Science
• Journal Name: ACM Transactions on Information Systems
• Publisher: ACM
• ISSN: 1046-8188
• ISSN (Online): 1558-2868
• Published Online: 26 May 2023
• Copyright Holders: © 2023 Copyright held by the owner/author(s).
• First Published: First published in ACM Transactions on Information Systems 2023
• Publisher Policy: Reproduced in accordance with the publisher copyright policy