Scaling Up Collaborative Filtering Data Sets through Randomized Fractal Expansions
arXiv:1905.09874
Abstract
Recommender system research suffers from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap, we propose to generate large-scale user/item interaction data sets by expanding pre-existing public data sets. Our key contribution is a technique that expands user/item incidence matrices matrices to large numbers of rows (users), columns (items), and non-zero values (interactions). The proposed method adapts Kronecker Graph Theory to preserve key higher order statistical properties such as the fat-tailed distribution of user engagements, item popularity, and singular value spectra of user/item interaction matrices. Preserving such properties is key to building large realistic synthetic data sets which in turn can be employed reliably to benchmark recommender systems and the systems employed to train them. We further apply our stochastic expansion algorithm to the binarized MovieLens 20M data set, which comprises 20M interactions between 27K movies and 138K users. The resulting expanded data set has 1.2B ratings, 2.2M users, and 855K items, which can be scaled up or down.
arXiv admin note: substantial text overlap with arXiv:1901.08910
References in corpus (11)
- scikit-image: Image processing in Python
- Robust dynamic classes revealed by measuring the response function of a social system
- Kronecker Graphs: An Approach to Modeling Networks
- How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility
- How To Break Anonymity of the Netflix Prize Dataset
- A Neural Autoregressive Approach to Collaborative Filtering
- Structural Deep Embedding for Hyper-Networks
- Deep Interest Network for Click-Through Rate Prediction
- A dual framework for low-rank tensor completion
- Scalable Realistic Recommendation Datasets through Fractal Expansions
- Human Interaction with Recommendation Systems