RecJPQ: Training Large-Catalogue Sequential Recommenders

Petrov, A. V. and Macdonald, C. (2024) RecJPQ: Training Large-Catalogue Sequential Recommenders. In: 17th ACM International Conference on Web Search and Data Mining, Merida, Mexico, 04-08 Mar 2024, pp. 538-547. ISBN 9798400703713 (doi: 10.1145/3616855.3635821)

Abstract

Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which replaces each item embedding with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenizers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48× reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularization effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer models in industrial applications, where datasets with millions of items are common.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Macdonald, Professor Craig and Petrov, Aleksandr
• Authors: Petrov, A. V., and Macdonald, C.
• College/School: College of Science and Engineering; College of Science and Engineering > School of Computing Science
• ISBN: 9798400703713
• Copyright Holders: Copyright © 2024 Copyright held by the owner/author(s)
• First Published: First published in WSDM '24: Proceedings of the 17th ACM International Conference on Web Search and Data Mining
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation