Building simulated queries for known-item topics: an analysis using six european languages

Azzopardi, L., de Rijke, M. and Balog, K. (2007) Building simulated queries for known-item topics: an analysis using six european languages. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, 23-27 July 2007, pp. 455-462. ISBN 9781595935977 (doi: 10.1145/1277741.1277820)

Abstract

There has been increased interest in the use of simulated queries for evaluation and estimation purposes in Information Retrieval. However, there are still many unaddressed issues regarding their usage and impact on evaluation because their quality, in terms of retrieval performance, is unlike real queries. In this paper, we focus on methods for building simulated known-item topics and explore their quality against real known-item topics. Using existing generation models as our starting point, we explore factors which may influence the generation of the known-item topic. Informed by this detailed analysis (on six European languages) we propose a model with improved document and term selection properties, showing that simulated known-item topics can be generated that are comparable to real known-item topics. This is a significant step towards validating the potential usefulness of simulated queries: for evaluation purposes, and because building models of querying behavior provides a deeper insight into the querying process so that better retrieval mechanisms can be developed to support the user.

• Item Type: Conference Proceedings
• Additional Information: © ACM, 2007. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval http://doi.acm.org/10.1145/1277741.1277820
• Keywords: Query simulation, query generations, evaluation, multilingual retrieval.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Azzopardi, Dr Leif
• Authors: Azzopardi, L., de Rijke, M., and Balog, K.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• College/School: College of Science and Engineering > School of Computing Science
• Publisher: ACM
• ISBN: 9781595935977
• Copyright Holders: Copyright © 2007 ACM
• First Published: First published in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher.