7 citations · 7 across the 6 of their papers we have counts for
6 papers
Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage
Hu Xu, Zhaolong Xing, Congcong Liu +5
Post-training LLM pruning relies on a small calibration set, usually chosen heuristically and judged by WikiText perplexity or by mean accuracy on general-domain commonsense benchm…
FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models
Zhida Jiang, Zhaolong Xing, Yang Pei +14
Industrial-grade distributed training of vision-language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design t…
The Hidden Power of Scaling Factor in LoRA Optimization
Zicheng Zhang, Haoran Li, Jiaxing Wang +10
In Low-Rank Adaptation (LoRA), the scaling factor is often treated as a mere complement to the learning rate, yet its role in optimization remains poorly understood. In this pa…
NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining
Zhida Jiang, Zhaolong Xing, Huichao Chai +12
Modern recommendation models have increased to trillions of parameters. As cluster scales expand to O(1k), distributed training bottlenecks shift from computation and memory to dat…
FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning
Zhida Jiang, Zhaolong Xing, Jiawei Lu +13
Despite algorithm-level innovations for multi-agent reinforcement learning (MARL), the underlying infrastructure for large-scale MARL remains underexplored. Existing training frame…
Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning
Zhida Jiang, Yang Xu, Hongli Xu +2
Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. However, the existing works fail to address all these practical concerns i…