Max-Utility Based Arm Selection Strategy for Sequential Query Recommendations

Puthiya Parambath, S. , Anagnostopoulos, C. , Murray-Smith, R. , MacAvaney, S. and Zervas, E. (2021) Max-Utility Based Arm Selection Strategy for Sequential Query Recommendations. In: 13th Asian Conference on Machine Learning (ACML 2021), 17-19 Nov 2021, pp. 564-579.

[img] Text
252868.pdf - Accepted Version


Publisher's URL:


We consider the query recommendation problem in closed loop interactive learning settings like online information gathering and exploratory analytics. The problem can be naturally modelled using the Multi-Armed Bandits (MAB) framework with countably many arms. The standard MAB algorithms for countably many arms begin with selecting a random set of candidate arms and then applying standard MAB algorithms, e.g., UCB, on this candidate set downstream. We show that such a selection strategy often results in higher cumulative regret and to this end, we propose a selection strategy based on the maximum utility of the arms. We show that in tasks like online information gathering, where sequential query recommendations are employed, the sequences of queries are correlated and the number of potentially optimal queries can be reduced to a manageable size by selecting queries with maximum utility with respect to the currently executing query. Our experimental results using a recent real online literature discovery service log file demonstrate that the proposed arm selection strategy improves the cumulative regret substantially with respect to the state-of-the-art baseline algorithms.

Item Type:Conference Proceedings
Additional Information:This work is partially funded by the UK EPSRC Grant Number: EP/R018634/1. We thank Allen Institute for providing the SemanticScholar query logs.
Glasgow Author(s) Enlighten ID:Murray-Smith, Professor Roderick and Anagnostopoulos, Dr Christos and MacAvaney, Dr Sean and Puthiya Parambath, Dr Sham
Authors: Puthiya Parambath, S., Anagnostopoulos, C., Murray-Smith, R., MacAvaney, S., and Zervas, E.
College/School:College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright © 2021 S.A. Puthiya Parambath, C. Anagnostopoulos, R. Murray-Smith, S. MacAvaney and E. Zervas.
First Published:First published in Proceedings of Machine Learning Research 157: 564-579
Publisher Policy:Reproduced in accordance with the publisher copyright policy
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
300982Exploiting Closed-Loop Aspects in Computationally and Data Intensive AnalyticsRoderick Murray-SmithEngineering and Physical Sciences Research Council (EPSRC)EP/R018634/1Computing Science