AutoLossGen: Automatic Loss Function Generation for Recommender Systems

Abstract

In recommendation systems, the choice of loss function is critical since a good loss may significantly improve the model performance. However, manually designing a good loss is a big challenge due to the complexity of the problem. A large fraction of previous work focuses on handcrafted loss functions, which needs significant expertise and human effort. In this paper, inspired by the recent development of automated machine learning, we propose an automatic loss function generation framework, AutoLossGen, which is able to generate loss functions directly constructed from basic mathematical operators without prior knowledge on loss structure. More specifically, we develop a controller model driven by reinforcement learning to generate loss functions, and develop iterative and alternating optimization schedule to update the parameters of both the controller model and the recommender model. One challenge for automatic loss generation in recommender systems is the extreme sparsity of recommendation datasets, which leads to the sparse reward problem for loss generation and search. To solve the problem, we further develop a reward filtering mechanism for efficient and effective loss generation. Experimental results show that our framework manages to create tailored loss functions for different recommendation models and datasets, and the generated loss gives better recommendation performance than commonly used baseline losses. Besides, most of the generated losses are transferable, i.e., the loss generated based on one model and dataset also works well for another model or dataset. Source code of the work is available at https://github.com/rutgerswiselab/AutoLossGen.

arXiv comment12 pages, 6 figures. Conference: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '22)

References in corpus (4)

• Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
• Neural Collaborative Reasoning
• Graph Collaborative Reasoning
• Learning Personalized Risk Preferences for Recommendation

Cited by in corpus (7)

• Explainable Fairness in Recommendation
• Exploration and Regularization of the Latent Action Space in Recommendation
• Graph-based Alignment and Uniformity for Recommendation
• Future Impact Decomposition in Request-level Recommendations
• Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS)
• Automated Contrastive Learning Strategy Search for Time Series
• Value Function Decomposition in Markov Recommendation Process