GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting
arXiv:2407.19682 · doi:10.1145/3637528.3671585
Abstract
Recommender systems require the simultaneous optimization of multiple objectives to accurately model user interests, necessitating the application of multi-task learning methods. However, existing multi-task learning methods in recommendations overlook the specific characteristics of recommendation scenarios, falling short in achieving proper gradient balance. To address this challenge, we set the target of multi-task learning as attaining the appropriate magnitude balance and the global direction balance, and propose an innovative methodology named GradCraft in response. GradCraft dynamically adjusts gradient magnitudes to align with the maximum gradient norm, mitigating interference from gradient magnitudes for subsequent manipulation. It then employs projections to eliminate gradient conflicts in directions while considering all conflicting tasks simultaneously, theoretically guaranteeing the global resolution of direction conflicts. GradCraft ensures the concurrent achievement of appropriate magnitude balance and global direction balance, aligning with the inherent characteristics of recommendation scenarios. Both offline and online experiments attest to the efficacy of GradCraft in enhancing multi-task performance in recommendations. The source code for GradCraft can be accessed at https://github.com/baiyimeng/GradCraft.
Accepted by KDD'24
References in corpus (9)
- Neural Graph Collaborative Filtering
- On the Effectiveness of Sampled Softmax Loss for Item Recommendation
- MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks
- Uncovering User Interest from Biased and Noised Watch Time in Video Recommendation
- Understanding and Modeling Passive-Negative Feedback for Short-video Sequential Recommendation
- Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation
- Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework
- Learning and Optimization of Implicit Negative Feedback for Industrial Short-video Recommender System
- LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting