A Study of Snippet Length and Informativeness: Behaviour, Performance and User Experience

Maxwell, D., Azzopardi, L. and Moshfeghi, Y. (2017) A Study of Snippet Length and Informativeness: Behaviour, Performance and User Experience. In: SIGIR 2017: The 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan, 7-11 Aug 2017, pp. 135-144. ISBN 9781450350228 (doi:10.1145/3077136.3080824)

[img]
Preview
Text
145412.pdf - Accepted Version

1MB

Abstract

The design and presentation of a Search Engine Results Page (SERP) has been subject to much research. With many contemporary aspects of the SERP now under scrutiny, work still remains in investigating more traditional SERP components, such as the result summary. Prior studies have examined a variety of different aspects of result summaries, but in this paper we investigate the influence of result summary length on search behaviour, performance and user experience. To this end, we designed and conducted a within-subjects experiment using the TREC AQUAINT news collection with 53 participants. Using Kullback-Leibler distance as a measure of information gain, we examined result summaries of different lengths and selected four conditions where the change in information gain was the greatest: (i) title only; (ii) title plus one snippet; (iii) title plus two snippets; and (iv) title plus four snippets. Findings show that participants broadly preferred longer result summaries, as they were perceived to be more informative. However, their performance in terms of correctly identifying relevant documents was similar across all four conditions. Furthermore, while the participants felt that longer summaries were more informative, empirical observations suggest otherwise; while participants were more likely to click on relevant items given longer summaries, they also were more likely to click on non-relevant items. This shows that longer is not necessarily better, though participants perceived that to be the case - and second, they reveal a positive relationship between the length and informativeness of summaries and their attractiveness (i.e. clickthrough rates). These findings show that there are tensions between perception and performance when designing result summaries that need to be taken into account.

Item Type:Conference Proceedings
Additional Information:The lead author is financially supported by the UK Government through the EPSRC, grant No. 1367507.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Azzopardi, Dr Leif and Maxwell, Mr David and Moshfeghi, Dr Yashar
Authors: Maxwell, D., Azzopardi, L., and Moshfeghi, Y.
College/School:College of Science and Engineering > School of Computing Science
Publisher:ACM Press
ISBN:9781450350228
Copyright Holders:Copyright © 2017 The Authors
First Published:First published in SIGIR 2017: The 40th International ACM SIGIR Conference on Research and Development in Information Retrieval: 135-144
Publisher Policy:Reproduced in accordance with the publisher copyright policy
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record