1 citations · 1 across the 2 of their papers we have counts for
7 papers
RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
Yilang Zhang, Bingcong Li, Georgios B. Giannakis
Low-Rank Adaptation (LoRA) lowers the computational and memory overhead of fine-tuning large models by updating a low-dimensional subspace of the pre-trained weight matrix. Albeit…
Surrogate modeling for Bayesian optimization beyond a single Gaussian process
Qin Lu, Konstantinos D. Polyzos, Bingcong Li +1
Bayesian optimization (BO) has well-documented merits for optimizing black-box functions with an expensive evaluation cost. Such functions emerge in applications as diverse as hype…
Low-Rank Adaptation Redux for Large Models
Bingcong Li, Yilang Zhang, Georgios B. Giannakis
Low-rank adaptation (LoRA) has emerged as the de facto standard for parameter-efficient fine-tuning (PEFT) of foundation models, enabling the adaptation of billion-parameter networ…
ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling
Yilang Zhang, Bingcong Li, Niao He +1
Scaling network depth has been a central driver behind the success of modern foundation models, yet recent investigations suggest that deep layers are often underutilized. This pap…
VASSO: Variance Suppression for Sharpness-Aware Minimization
Bingcong Li, Yilang Zhang, Georgios B. Giannakis
Sharpness-aware minimization (SAM) has well-documented merits in enhancing generalization of deep neural network models. Accounting for sharpness in the loss function geometry, whe…
Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning
Yilang Zhang, Bingcong Li, Georgios B. Giannakis
Utilizing task-invariant knowledge acquired from related tasks as prior information, meta-learning offers a principled approach to learning a new task with limited data records. Sa…