White-box: on the prediction of collaborative filtering recommendation systems' performance

Paun, I., Moshfeghi, Y. and Ntarmos, N. (2023) White-box: on the prediction of collaborative filtering recommendation systems' performance. ACM Transactions on Internet Technology, 23(1), 8. (doi: 10.1145/3554979)

Abstract

Collaborative Filtering recommendation algorithms (CF) are a popular solution to the information overload problem, aiding users in the item selection process. Relevant research has long focused on refining and improving these models to produce better (more effective) recommendations, and has converged on a methodology to predict their effectiveness on target datasets by evaluating them on random samples of the latter. However, predicting the efficiency of the solutions – especially with regards to their time- and resource-hungry training phase, whose requirements dwarf those of the prediction/recommendation phase – has received little to no attention in the literature. This paper addresses this gap for a number of representative and highly popular CF models, including algorithms based on matrix factorisation, k-nearest neighbours, co-clustering, and slope one schemes. To this end, we first study the computational complexity of the training phase of said CF models and derive time and space complexity equations. Then, using characteristics of the input and the aforementioned equations, we contribute a methodology for predicting the processing time and memory usage of their training phase. Our contributions further include an adaptive sampling strategy, to address the trade-off between resource usage costs and prediction accuracy, and a framework which quantifies both the efficiency and effectiveness of CF. Finally, a systematic experimental evaluation demonstrates that our method outperforms state-of-the-art regression schemes by a considerable margin, with an overhead that is a small fraction of the overall requirements of CF training.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Ntarmos, Dr Nikos and Moshfeghi, Dr Yashar and Paun, Ms Iulia
• Authors: Paun, I., Moshfeghi, Y., and Ntarmos, N.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: ACM Transactions on Internet Technology
• Publisher: Association for Computing Machinery
• ISSN: 1533-5399
• ISSN (Online): 1557-6051
• Published Online: 12 August 2022
• Copyright Holders: Copyright © 2022 Association for Computing Machinery.
• First Published: First published in ACM Transactions on Internet Technology 23(1): 8
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
172865		EPSRC DTP 16/17 and 17/18	Mary Beth Kneafsey	Engineering and Physical Sciences Research Council (EPSRC)	EP/N509668/1	Research and Innovation Services