On Single and Multiple Representations in Dense Passage Retrieval

Macdonald, C. , Tonellotto, N. and Ounis, I. (2021) On Single and Multiple Representations in Dense Passage Retrieval. In: 11th Italian Information Retrieval Workshop (IIR 2021), Bari, Italy, 13-15 Sep 2021,

[img] Text
248964.pdf - Published Version
Available under License Creative Commons Attribution.

779kB

Publisher's URL: http://ceur-ws.org/Vol-2947/

Abstract

The advent of contextualised language models has brought gains in search effectiveness, not just when applied for re-ranking the output of classical weighting models such as BM25, but also when used directly for passage indexing and retrieval, a technique which is called dense retrieval. In the existing literature in neural ranking, two dense retrieval families have become apparent: single representation, where entire passages are represented by a single embedding (usually BERT’s [CLS] token, as exemplified by the recent ANCE approach), or multiple representations, where each token in a passage is represented by its own embedding (as exemplified by the recent ColBERT approach). These two families have not been directly compared. However, because of the likely importance of dense retrieval moving forward, a clear understanding of their advantages and disadvantages is paramount. To this end, this paper contributes a direct study on their comparative effectiveness, noting situations where each method under/over performs w.r.t. each other, and w.r.t. a BM25 baseline. We observe that, while ANCE is more efficient than ColBERT in terms of response time and memory usage, multiple representations are statistically more effective than the single representations for MAP and MRR@10. We also show that multiple representations get better improvements than single representations for queries being the hardest for BM25, as well as for definitional queries, and those with complex information needs.

Item Type:Conference Proceedings
Additional Information:Nicola Tonellotto was partially supported by the Italian Ministry of Education and Research (MIUR) in the framework of the CrossLab project (Departments of Excellence). Craig Macdonald and Iadh Ounis acknowledge EPSRC grant EP/ R018634/1: Closed-Loop Data Science for Complex, Computationally- & Data-Intensive Analytics.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Tonellotto, Dr Nicola and Ounis, Professor Iadh and Macdonald, Dr Craig
Authors: Macdonald, C., Tonellotto, N., and Ounis, I.
College/School:College of Science and Engineering > School of Computing Science
ISSN:1613-0073
Published Online:19 September 2021
Copyright Holders:© 2021 Copyright for this paper by its authors
First Published:First published in Proceedings of the 11th Italian Information Retrieval Workshop 2021, Vol. 2947
Publisher Policy:Reproduced under a Creative Commons licence
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
300982Exploiting Closed-Loop Aspects in Computationally and Data Intensive AnalyticsRoderick Murray-SmithEngineering and Physical Sciences Research Council (EPSRC)EP/R018634/1Computing Science