Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems

Abstract

In recent years, algorithm research in the area of recommender systems has shifted from matrix factorization techniques and their latent factor models to neural approaches. However, given the proven power of latent factor models, some newer neural approaches incorporate them within more complex network architectures. One specific idea, recently put forward by several researchers, is to consider potential correlations between the latent factors, i.e., embeddings, by applying convolutions over the user-item interaction map. However, contrary to what is claimed in these articles, such interaction maps do not share the properties of images where Convolutional Neural Networks (CNNs) are particularly useful. In this work, we show through analytical considerations and empirical evaluations that the claimed gains reported in the literature cannot be attributed to the ability of CNNs to model embedding correlations, as argued in the original papers. Moreover, additional performance evaluations show that all of the examined recent CNN-based models are outperformed by existing non-neural machine learning techniques or traditional nearest-neighbor approaches. On a more general level, our work points to major methodological issues in recommender systems research.

arXiv commentSource code available here: https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation

References in corpus (8)

• BPR: Bayesian Personalized Ranking from Implicit Feedback
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Evaluation of Session-based Recommendation Algorithms
• On the Difficulty of Evaluating Baselines: A Study on Recommender Systems
• Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models
• Artist-driven layering and user's behaviour impact on recommendations in a playlist continuation scenario

Cited by in corpus (6)

• A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research
• CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention
• Neural Collaborative Reasoning
• Reenvisioning Collaborative Filtering vs Matrix Factorization
• An Evaluation Study of Generative Adversarial Networks for Collaborative Filtering
• Reproducibility and Artifact Consistency of the SIGIR 2022 Recommender Systems Papers Based on Message Passing