An Evaluation Study of Generative Adversarial Networks for Collaborative Filtering

Abstract

This work explores the reproducibility of CFGAN. CFGAN and its family of models (TagRec, MTPR, and CRGAN) learn to generate personalized and fake-but-realistic rankings of preferences for top-N recommendations by using previous interactions. This work successfully replicates the results published in the original paper and discusses the impact of certain differences between the CFGAN framework and the model used in the original evaluation. The absence of random noise and the use of real user profiles as condition vectors leaves the generator prone to learn a degenerate solution in which the output vector is identical to the input vector, therefore, behaving essentially as a simple autoencoder. The work further expands the experimental analysis comparing CFGAN against a selection of simple and well-known properly optimized baselines, observing that CFGAN is not consistently competitive against them despite its high computational cost. To ensure the reproducibility of these analyses, this work describes the experimental methodology and publishes all datasets and source code.

References in corpus (10)

• Generative Adversarial Networks: An Overview
• An Introduction to Variational Autoencoders
• Solving the apparent diversity-accuracy dilemma of recommender systems
• IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
• Evaluation of Session-based Recommendation Algorithms
• Embarrassingly Shallow Autoencoders for Sparse Data
• A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research
• Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models
• Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems