4 papers
Feed m Birds with One Scone: Accelerating Multi-task Gradient Balancing via Bi-level Optimization
Xuxing Chen, Yun He, Jiayi Xu +9
In machine learning, the goal of multi-task learning (MTL) is to optimize multiple objectives together. Recent works, for example, Multiple Gradient Descent Algorithm (MGDA) and it…
Unifying Generative and Dense Retrieval for Sequential Recommendation
Liu Yang, Fabian Paischer, Kaveh Hassani +11
Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user thro…
APOLLO: SGD-like Memory, AdamW-level Performance
Hanqing Zhu, Zhenyu Zhang, Wenyan Cong +7
Large language models (LLMs) are notoriously memory-intensive during training, particularly with the popular AdamW optimizer. This memory burden necessitates using more or higher-e…
MultiBalance: Multi-Objective Gradient Balancing in Industrial-Scale Multi-Task Recommendation System
Yun He, Xuxing Chen, Jiayi Xu +11
In industrial recommendation systems, multi-task learning (learning multiple tasks simultaneously on a single model) is a predominant approach to save training/serving resources an…