The impact of result diversification on search behaviour and performance

Maxwell, D. M., Azzopardi, L. and Moshfeghi, Y. (2019) The impact of result diversification on search behaviour and performance. Information Retrieval, 22, pp. 422-446. (doi: 10.1007/s10791-019-09353-0)

[img] Text
188766.pdf - Published Version
Available under License Creative Commons Attribution.



Result diversification aims to provide searchers with a broader view of a given topic while attempting to maximise the chances of retrieving relevant material. Diversifying results also aims to reduce search bias by increasing the coverage over different aspects of the topic. As such, searchers should learn more about the given topic in general. Despite diversification algorithms being introduced over two decades ago, little research has explicitly examined their impact on search behaviour and performance in the context of Interactive Information Retrieval (IIR). In this paper, we explore the impact of diversification when searchers undertake complex search tasks that require learning about different aspects of a topic (aspectual retrieval). We hypothesise that by diversifying search results, searchers will be exposed to a greater number of aspects. In turn, this will maximise their coverage of the topic (and thus reduce possible search bias). As a consequence, diversification should lead to performance benefits, regardless of the task, but how does diversification affect search behaviours and search satisfaction? Based on Information Foraging Theory (IFT), we infer two hypotheses regarding search behaviours due to diversification, namely that (i) it will lead to searchers examining fewer documents per query, and (ii) it will also mean searchers will issue more queries overall. To this end, we performed a within-subjects user study using the TREC AQUAINT collection with 51 participants, examining the differences in search performance and behaviour when using (i) a non-diversified system (BM25) versus (ii) a diversified system (BM25+xQuAD) when the search task is either (a) ad-hoc or (b) aspectual. Our results show a number of notable findings in terms of search behaviour: participants on the diversified system issued more queries and examined fewer documents per query when performing the aspectual search task. Furthermore, we showed that when using the diversified system, participants were: more successful in marking relevant documents, and obtained a greater awareness of the topics (i.e. identified relevant documents containing more novel aspects). These findings show that search behaviour is influenced by diversification and task complexity. They also motivate further research into complex search tasks such as aspectual retrieval -- and how diversity can play an important role in improving the search experience, by providing greater coverage of a topic and mitigating potential bias in search results.

Item Type:Articles
Additional Information:The lead author acknowledges support from the EPSRC under Grant Number 1367507.
Keywords:diversification, user study, user behaviour, user behavior, search performance, aspectual retrieval
Glasgow Author(s) Enlighten ID:MAXWELL, David Martin and Azzopardi, Dr Leif and Moshfeghi, Dr Yashar
Authors: Maxwell, D. M., Azzopardi, L., and Moshfeghi, Y.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
College/School:College of Science and Engineering
College of Science and Engineering > School of Computing Science
Research Group:Information Retrieval
Journal Name:Information Retrieval
ISSN (Online):1573-7659
Published Online:16 May 2019
Copyright Holders:Copyright © 2019 The Author(s)
First Published:First published in Information Retrieval 22:422–446
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
608951Engineering and Physical Sciences Doctoral Training Grant 2012-16Mary Beth KneafseyEngineering and Physical Sciences Research Council (EPSRC)EP/K503058/1VPO VICE PRINCIPAL RESEARCH & ENTERPRISE
701101EPSRC 2015 DTPMary Beth KneafseyEngineering and Physical Sciences Research Council (EPSRC)EP/M508056/1R&I - RESEARCH STRATEGY & INNOVATION