Training Deep AutoEncoders for Collaborative Filtering

Abstract

This paper proposes a novel model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically demonstrate that: a) deep autoencoder models generalize much better than the shallow ones, b) non-linear activation functions with negative parts are crucial for training deep models, and c) heavy use of regularization techniques such as dropout is necessary to prevent over-fiting. We also propose a new training algorithm based on iterative output re-feeding to overcome natural sparseness of collaborate filtering. The new algorithm significantly speeds up training and improves model performance. Our code is available at https://github.com/NVIDIA/DeepRecommender

5 pages, 6 figures

References in corpus (4)

• Empirical Analysis of Predictive Algorithms for Collaborative Filtering
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Self-Normalizing Neural Networks

Cited by in corpus (16)

• Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
• Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
• Multi-Task Graph Autoencoders
• Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks
• Creating A Neural Pedagogical Agent by Jointly Learning to Review and Assess
• Deep Learning Approach for Matrix Completion Using Manifold Learning
• Time-based Sequence Model for Personalization and Recommendation Systems
• Extendable Neural Matrix Completion
• Latent User Linking for Collaborative Cross Domain Recommendation
• Clinical Recommender System: Predicting Medical Specialty Diagnostic Choices with Neural Network Ensembles
• Wasserstein Autoencoders for Collaborative Filtering
• Matrix Factorization via Deep Learning
• Computing recommendations via a Knowledge Graph-aware Autoencoder
• Decentralized Multi-Target Cross-Domain Recommendation for Multi-Organization Collaborations
• Intelligent EHRs: Predicting Procedure Codes From Diagnosis Codes
• Attentive Autoencoders for Multifaceted Preference Learning in One-class Collaborative Filtering