32 citations · 51 across the 11 of their papers we have counts for
4 papers · 1 filter
Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature
Zihan Zhu, Zhehang Du, Xuyang Chen +5
Low-Rank Adaptation (LoRA) is an effective approach for adapting large pretrained models by learning low-rank weight updates. In practice, the LoRA rank is used to control an adapt…
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
Tim Tsz-Kit Lau, Weijian Li, Chenwei Xu +2
An appropriate choice of batch sizes in large-scale model training is crucial, yet it involves an intrinsic yet inevitable dilemma: large-batch training improves training efficienc…
AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
Tim Tsz-Kit Lau, Han Liu, Mladen Kolar
The choice of batch size in minibatch stochastic gradient optimization is critical for both optimization and generalization performance in large-scale model training. Although larg…
The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applications
Tim Tsz-Kit Lau, Biswa Sengupta
We study two state-of-the-art solutions to the multi-agent pickup and delivery (MAPD) problem based on different principles -- multi-agent path-finding (MAPF) and multi-agent reinf…