AutoLoss: Automated Loss Function Search in Recommendations

Abstract

Designing an effective loss function plays a crucial role in training deep recommender systems. Most existing works often leverage a predefined and fixed loss function that could lead to suboptimal recommendation quality and training efficiency. Some recent efforts rely on exhaustively or manually searched weights to fuse a group of candidate loss functions, which is exceptionally costly in computation and time. They also neglect the various convergence behaviors of different data examples. In this work, we propose an AutoLoss framework that can automatically and adaptively search for the appropriate loss function from a set of candidates. To be specific, we develop a novel controller network, which can dynamically adjust the loss probabilities in a differentiable manner. Unlike existing algorithms, the proposed controller can adaptively generate the loss probabilities for different data examples according to their varied convergence behaviors. Such design improves the model's generalizability and transferability between deep recommender systems and datasets. We evaluate the proposed framework on two benchmark datasets. The results show that AutoLoss outperforms representative baselines. Further experiments have been conducted to deepen our understandings of AutoLoss, including its transferability, components and training efficiency.

arXiv commentProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

References in corpus (23)

• Categorical Reparameterization with Gumbel-Softmax
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Deep Reinforcement Learning for Page-wise Recommendations
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
• SphereReID: Deep Hypersphere Manifold Embedding for Person Re-Identification
• Towards Long-term Fairness in Recommendation
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Neural Interactive Collaborative Filtering
• Jointly Learning to Recommend and Advertise
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
• Loss Function Search for Face Recognition
• AutoLoss: Learning Discrete Schedules for Alternate Optimization
• Distance Map Loss Penalty Term for Semantic Segmentation
• AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations
• Automated Self-Supervised Learning for Graphs
• Differentiable Neural Input Search for Recommender Systems
• Learnable Embedding Sizes for Recommender Systems
• Memory-efficient Embedding for Recommendations
• Toward Simulating Environments in Reinforcement Learning Based Recommendations
• Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation
• DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems
• Attacking Black-box Recommendations via Copying Cross-domain User Profiles