1 citations · 2 across the 9 of their papers we have counts for
3 papers · 1 filter
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
Tim Tsz-Kit Lau, Weijian Li, Chenwei Xu +2
An appropriate choice of batch sizes in large-scale model training is crucial, yet it involves an intrinsic yet inevitable dilemma: large-batch training improves training efficienc…
Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies
Yibo Wen, Chenwei Xu, Jerry Yao-Chieh Hu +2
We present a three-stage framework for training deep learning models specializing in antibody sequence-structure co-design. We first pre-train a language model using millions of an…
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
Tim Tsz-Kit Lau, Weijian Li, Chenwei Xu +2
Modern deep neural networks often require distributed training with many workers due to their large size. As the number of workers increases, communication overheads become the mai…