3 papers
cs.LG2025
Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
Hao Ban, Kaiyi Ji
Large language models are often adapted using parameter-efficient techniques such as Low-Rank Adaptation (LoRA), formulated as , where is the pre-trained para…
cs.LG2025
SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation
Hao Ban, Gokul Ram Subramani, Kaiyi Ji
Multi-task learning (MTL) enables a joint model to capture commonalities across multiple tasks, reducing computation costs and improving data efficiency. However, a major challenge…
cs.LG2024
Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis
Yifan Yang, Hao Ban, Minhui Huang +2
Bilevel optimization has recently attracted considerable attention due to its abundant applications in machine learning problems. However, existing methods rely on prior knowledge…