On Single and Multiple Representations in Dense Passage Retrieval

Macdonald, C. , Tonellotto, N. and Ounis, I. (2021) On Single and Multiple Representations in Dense Passage Retrieval. In: 11th Italian Information Retrieval Workshop (IIR 2021), Bari, Italy, 13-15 Sep 2021,

Abstract

The advent of contextualised language models has brought gains in search effectiveness, not just when applied for re-ranking the output of classical weighting models such as BM25, but also when used directly for passage indexing and retrieval, a technique which is called dense retrieval. In the existing literature in neural ranking, two dense retrieval families have become apparent: single representation, where entire passages are represented by a single embedding (usually BERT’s [CLS] token, as exemplified by the recent ANCE approach), or multiple representations, where each token in a passage is represented by its own embedding (as exemplified by the recent ColBERT approach). These two families have not been directly compared. However, because of the likely importance of dense retrieval moving forward, a clear understanding of their advantages and disadvantages is paramount. To this end, this paper contributes a direct study on their comparative effectiveness, noting situations where each method under/over performs w.r.t. each other, and w.r.t. a BM25 baseline. We observe that, while ANCE is more efficient than ColBERT in terms of response time and memory usage, multiple representations are statistically more effective than the single representations for MAP and MRR@10. We also show that multiple representations get better improvements than single representations for queries being the hardest for BM25, as well as for definitional queries, and those with complex information needs.

• Item Type: Conference Proceedings
• Additional Information: Nicola Tonellotto was partially supported by the Italian Ministry of Education and Research (MIUR) in the framework of the CrossLab project (Departments of Excellence). Craig Macdonald and Iadh Ounis acknowledge EPSRC grant EP/ R018634/1: Closed-Loop Data Science for Complex, Computationally- & Data-Intensive Analytics.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Macdonald, Professor Craig and Ounis, Professor Iadh and Tonellotto, Dr Nicola
• Authors: Macdonald, C., Tonellotto, N., and Ounis, I.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 1613-0073
• Published Online: 19 September 2021
• Copyright Holders: © 2021 Copyright for this paper by its authors
• First Published: First published in Proceedings of the 11th Italian Information Retrieval Workshop 2021, Vol. 2947
• Publisher Policy: Reproduced under a Creative Commons licence

Related URLs

• Organisation

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
300982		Exploiting Closed-Loop Aspects in Computationally and Data Intensive Analytics	Roderick Murray-Smith	Engineering and Physical Sciences Research Council (EPSRC)	EP/R018634/1	Computing Science