Learning to predict response times for online query scheduling

Macdonald, C., Ounis, I. and Tonellotto, N. (2012) Learning to predict response times for online query scheduling. In: 35th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR 2012, Portland OR, USA, 12-16 Aug 2012, (doi:10.1145/2348283.2348367)

Full text not currently available from Enlighten.

Publisher's URL: http://dx.doi.org/10.1145/2348283.2348367

Abstract

Dynamic pruning strategies permit efficient retrieval by not fully scoring all postings of the documents matching a query -- without degrading the retrieval effectiveness of the top-ranked results. However, the amount of pruning achievable for a query can vary, resulting in queries taking different amounts of time to execute. Knowing in advance the execution time of queries would permit the exploitation of online algorithms to schedule queries across replicated servers in order to minimise the average query waiting and completion times. In this work, we investigate the impact of dynamic pruning strategies on query response times, and propose a framework for predicting the efficiency of a query. Within this framework, we analyse the accuracy of several query efficiency predictors across 10,000 queries submitted to in-memory inverted indices of a 50-million-document Web crawl. Our results show that combining multiple efficiency predictors with regression can accurately predict the response times of a query before it is executed. Moreover, using the efficiency predictors to facilitate online scheduling algorithms can result in a 25% reduction in the average waiting time experienced by queries before processing.

Item Type:Conference Proceedings
Additional Information:9781450314725
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Macdonald, Dr Craig and Ounis, Professor Iadh
Authors: Macdonald, C., Ounis, I., and Tonellotto, N.
College/School:College of Science and Engineering > School of Computing Science

University Staff: Request a correction | Enlighten Editors: Update this record