How to Retrain Recommender System? A Sequential Meta-Learning Method

Abstract

Practical recommender systems need be periodically retrained to refresh the model with new interaction data. To pursue high model fidelity, it is usually desirable to retrain the model on both historical and new data, since it can account for both long-term and short-term user preference. However, a full model retraining could be very time-consuming and memory-costly, especially when the scale of historical data is large. In this work, we study the model retraining mechanism for recommender systems, a topic of high practical values but has been relatively little explored in the research community. Our first belief is that retraining the model on historical data is unnecessary, since the model has been trained on it before. Nevertheless, normal training on new data only may easily cause overfitting and forgetting issues, since the new data is of a smaller scale and contains fewer information on long-term user preference. To address this dilemma, we propose a new training method, aiming to abandon the historical data during retraining through learning to transfer the past training experience. Specifically, we design a neural network-based transfer component, which transforms the old model to a new model that is tailored for future recommendations. To learn the transfer component well, we optimize the "future performance" -- i.e., the recommendation accuracy evaluated in the next time period. Our Sequential Meta-Learning(SML) method offers a general training paradigm that is applicable to any differentiable model. We demonstrate SML on matrix factorization and conduct experiments on two real-world datasets. Empirical results show that SML not only achieves significant speed-up, but also outperforms the full model retraining in recommendation accuracy, validating the effectiveness of our proposals. We release our codes at: https://github.com/zyang1580/SML.

arXiv commentAppear in SIGIR 2020

References in corpus (7)

• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Neural Graph Collaborative Filtering
• KGAT: Knowledge Graph Attention Network for Recommendation
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Bilevel Programming for Hyperparameter Optimization and Meta-Learning
• LambdaOpt: Learn to Regularize Recommender Models in Finer Levels

Cited by in corpus (12)

• CATN: Cross-Domain Recommendation for Cold-Start Users via Aspect Transfer Network
• AutoDebias: Learning to Debias for Recommendation
• Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
• A Critical Study on Data Leakage in Recommender System Offline Evaluation
• Cross-Market Product Recommendation
• Recommendation Unlearning via Influence Function
• INMO: A Model-Agnostic and Scalable Module for Inductive Collaborative Filtering
• Online Caching with Optimistic Learning
• Online Item Cold-Start Recommendation with Popularity-Aware Meta-Learning
• LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting
• Mitigating Divergence of Latent Factors via Dual Ascent for Low Latency Event Prediction Models
• Improving conversion rate prediction via self-supervised pre-training in online advertising