Maxwell, D. M., Azzopardi, L. and Moshfeghi, Y. (2019) The impact of result diversification on search behaviour and performance. Information Retrieval, 22, pp. 422-446. (doi: 10.1007/s10791-019-09353-0)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
188766.pdf - Published Version Available under License Creative Commons Attribution. 1MB |
Abstract
Result diversification aims to provide searchers with a broader view of a given topic while attempting to maximise the chances of retrieving relevant material. Diversifying results also aims to reduce search bias by increasing the coverage over different aspects of the topic. As such, searchers should learn more about the given topic in general. Despite diversification algorithms being introduced over two decades ago, little research has explicitly examined their impact on search behaviour and performance in the context of Interactive Information Retrieval (IIR). In this paper, we explore the impact of diversification when searchers undertake complex search tasks that require learning about different aspects of a topic (aspectual retrieval). We hypothesise that by diversifying search results, searchers will be exposed to a greater number of aspects. In turn, this will maximise their coverage of the topic (and thus reduce possible search bias). As a consequence, diversification should lead to performance benefits, regardless of the task, but how does diversification affect search behaviours and search satisfaction? Based on Information Foraging Theory (IFT), we infer two hypotheses regarding search behaviours due to diversification, namely that (i) it will lead to searchers examining fewer documents per query, and (ii) it will also mean searchers will issue more queries overall. To this end, we performed a within-subjects user study using the TREC AQUAINT collection with 51 participants, examining the differences in search performance and behaviour when using (i) a non-diversified system (BM25) versus (ii) a diversified system (BM25+xQuAD) when the search task is either (a) ad-hoc or (b) aspectual. Our results show a number of notable findings in terms of search behaviour: participants on the diversified system issued more queries and examined fewer documents per query when performing the aspectual search task. Furthermore, we showed that when using the diversified system, participants were: more successful in marking relevant documents, and obtained a greater awareness of the topics (i.e. identified relevant documents containing more novel aspects). These findings show that search behaviour is influenced by diversification and task complexity. They also motivate further research into complex search tasks such as aspectual retrieval -- and how diversity can play an important role in improving the search experience, by providing greater coverage of a topic and mitigating potential bias in search results.
| Item Type: | Articles |
|---|---|
| Additional Information: | The lead author acknowledges support from the EPSRC under Grant Number 1367507. |
| Keywords: | diversification, user study, user behaviour, user behavior, search performance, aspectual retrieval |
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | MAXWELL, David Martin and Azzopardi, Dr Leif and Moshfeghi, Dr Yashar |
| Authors: | Maxwell, D. M., Azzopardi, L., and Moshfeghi, Y. |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science |
| College/School: | College of Science and Engineering College of Science and Engineering > School of Computing Science |
| Research Group: | Information Retrieval |
| Journal Name: | Information Retrieval |
| Publisher: | Springer |
| ISSN: | 1386-4564 |
| ISSN (Online): | 1573-7659 |
| Published Online: | 16 May 2019 |
| Copyright Holders: | Copyright © 2019 The Author(s) |
| First Published: | First published in Information Retrieval 22:422–446 |
| Publisher Policy: | Reproduced under a Creative Commons License |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information