gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling

Petrov, A. V. and Macdonald, C. (2023) gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling. In: 17th ACM Conference on Recommender Systems (RecSys 2023), Singapore, 18-22 Sept 2023, pp. 116-128. ISBN 9798400702419 (doi: 10.1145/3604915.3608783)

Abstract

A large catalogue size is one of the central challenges in training recommendation models: a large number of items makes them memory and computationally inefficient to compute scores for all items during training, forcing these models to deploy negative sampling. However, negative sampling increases the proportion of positive interactions in the training data, and therefore models trained with negative sampling tend to overestimate the probabilities of positive interactions – a phenomenon we call overconfidence. While the absolute values of the predicted scores/probabilities are not important for the ranking of retrieved recommendations, overconfident models may fail to estimate nuanced differences in the top-ranked items, resulting in degraded performance. In this paper, we show that overconfidence explains why the popular SASRec model underperforms when compared to BERT4Rec. This is contrary to the BERT4Rec authors’ explanation that the difference in performance is due to the bi-directional attention mechanism. To mitigate overconfidence, we propose a novel Generalised Binary Cross-Entropy Loss function (gBCE) and theoretically prove that it can mitigate overconfidence. We further propose the gSASRec model, an improvement over SASRec that deploys an increased number of negatives and the gBCE loss. We show through detailed experiments on three datasets that gSASRec does not exhibit the overconfidence problem. As a result, gSASRec can outperform BERT4Rec (e.g. +9.47% NDCG on the MovieLens-1M dataset), while requiring less training time (e.g. -73% training time on MovieLens-1M). Moreover, in contrast to BERT4Rec, gSASRec is suitable for large datasets that contain more than 1 million items.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Macdonald, Professor Craig and Petrov, Aleksandr
• Authors: Petrov, A. V., and Macdonald, C.
• College/School: College of Science and Engineering; College of Science and Engineering > School of Computing Science
• Research Centre: College of Science and Engineering > School of Computing Science > IDA Section > GPU Cluster
• ISBN: 9798400702419
• Copyright Holders: © 2023 Copyright held by the owner/author(s)
• First Published: First published in RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation