Naseri, S., Dalton, J. , Yates, A. and Allan, J. (2021) CEQE: Contextualized Embeddings for Query Expansion. In: 43rd European Conference on IR Research, ECIR 2021, 28 Mar - 1 Apr 2021, ISBN 9783030721121 (doi: 10.1007/978-3-030-72113-8_31)
Text
237968.pdf - Accepted Version 297kB |
Abstract
In this work we leverage recent advances in context-sensitive language models to improve the task of query expansion. Contextualized word representation models, such as ELMo and BERT, are rapidly replacing static embedding models. We propose a new model, Contextualized Embeddings for Query Expansion (CEQE), that utilizes query-focused contextualized embedding vectors. We study the behavior of contextual representations generated for query expansion in ad-hoc document retrieval. We conduct our experiments on probabilistic retrieval models as well as in combination with neural ranking models. We evaluate CEQE on two standard TREC collections: Robust and Deep Learning. We find that CEQE outperforms static embedding-based expansion methods on multiple collections (by up to 18% on Robust and 31% on Deep Learning on average precision) and also improves over proven probabilistic pseudo-relevance feedback (PRF) models. We further find that multiple passes of expansion and reranking result in continued gains in effectiveness with CEQE-based approaches outperforming other approaches. The final model incorporating neural and CEQE-based expansion score achieves gains of up to 5% in P@20 and 2% in AP on Robust over the state-of-the-art transformer-based re-ranking model, Birch.
Item Type: | Conference Proceedings |
---|---|
Status: | Published |
Refereed: | Yes |
Glasgow Author(s) Enlighten ID: | Dalton, Dr Jeff |
Authors: | Naseri, S., Dalton, J., Yates, A., and Allan, J. |
College/School: | College of Science and Engineering > School of Computing Science |
ISBN: | 9783030721121 |
Copyright Holders: | Copyright © 2021 Springer International Publishing |
First Published: | First published in Advances in Information Retrieval Part I, LCNS 12656 |
Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
University Staff: Request a correction | Enlighten Editors: Update this record