Self-Supervised Reinforcement Learning for Recommender Systems

Xin, X., Karatzoglou, A., Arapakis, I. and Jose, J. M. (2020) Self-Supervised Reinforcement Learning for Recommender Systems. In: 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020), Xi'an, China, 25-30 Jul 2020, ISBN 9781450380164 (doi: 10.1145/3397271.3401147)

Full text not currently available from Enlighten.

Abstract

In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, multiple types of user-item interactions such as clicks, purchases etc. The current state-of-the-art supervised approaches fail to model them appropriately. Casting sequential recommendation task as a reinforcement learning (RL) problem is a promising direction. A major component of RL approaches is to train the agent through interactions with the environment. However, it is often problematic to train a recommender in an on-line fashion due to the requirement to expose users to irrelevant recommendations. As a result, learning the policy from logged implicit feedback is of vital importance, which is challenging due to the pure off-policy setting and lack of negative rewards (feedback). In this paper, we propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. The RL part acts as a regularizer to drive the supervised layer focusing on specific rewards (e.g., recommending items which may lead to purchases rather than clicks) while the self-supervised layer with cross-entropy loss provides strong gradient signals for parameter updates. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning (SQN) and Self-Supervised Actor-Critic (SAC). We integrate the proposed frameworks with four state-of-the-art recommendation models. Experimental results on two real-world datasets demonstrate the effectiveness of our approach.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Jose, Professor Joemon and Xin, Xin
Authors: Xin, X., Karatzoglou, A., Arapakis, I., and Jose, J. M.
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Publisher:ACM
ISBN:9781450380164

University Staff: Request a correction | Enlighten Editors: Update this record