10 papers
Regularity-Aware Stochastic MGDA with Adaptive Per-Iterate Conflict-Avoidant Update
Chentong Huang, Lisha Chen
Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously. The multi-gradient descent algorithm (MGDA) constructs a common descent or conflict-avoidant (CA…
Improved Convergence Rates for Stochastic Multi-Gradient Descent which Close the Gap: A Proof by AI
Lisha Chen
For smooth nonconvex stochastic multi-objective optimization, stochastic multi-gradient descent (SMG) computes an approximate steepest common descent direction of the objectives fr…
SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Front
Liuyuan Jiang, Chentong Huang, Lisha Chen
Scalarization is widely used in multi-objective optimization owing to its simplicity and scalability. In many applications, the goal is to generate solutions that represent diverse…
Efficient Penalty-Based Bilevel Methods: Improved Analysis, Novel Updates, and Flatness Condition
Liuyuan Jiang, Quan Xiao, Lisha Chen +1
Penalty-based methods have become popular for solving bilevel optimization (BLO) problems, thanks to their effective first-order nature. However, they often require inner-loop iter…
BiRQ: Bi-Level Self-Labeling Random Quantization for Self-Supervised Speech Recognition
Liuyuan Jiang, Xiaodong Cui, Brian Kingsbury +2
Speech is a rich signal, and labeled audio-text pairs are costly, making self-supervised learning essential for scalable representation learning. A core challenge in speech SSL is…
Objective Soups: Multilingual Multi-Task Modeling for Speech Processing
A F M Saif, Lisha Chen, Xiaodong Cui +3
Training a single model for multilingual, multi-task speech processing (MSP) is severely hampered by conflicting objectives between tasks like speech recognition and translation. W…
Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions
Liuyuan Jiang, Quan Xiao, Lisha Chen +1
Bilevel optimization, a hierarchical optimization paradigm, has gained significant attention in a wide range of practical applications, notably in the fine-tuning of generative mod…
Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance
Lisha Chen, Quan Xiao, Ellen Hidemi Fukuda +3
Multi-objective learning under user-specified preference is common in real-world problems such as multi-lingual speech recognition under fairness. In this work, we frame such a pro…
Bilevel Joint Unsupervised and Supervised Training for Automatic Speech Recognition
Xiaodong Cui, A F M Saif, Songtao Lu +4
In this paper, we propose a bilevel joint unsupervised and supervised training (BL-JUST) framework for automatic speech recognition. Compared to the conventional pre-training and f…
FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning
Lisha Chen, AFM Saif, Yanning Shen +1
Finding specific preference-guided Pareto solutions that represent different trade-offs among multiple objectives is critical yet challenging in multi-objective problems. Existing…