2 citations · 2 across the 15 of their papers we have counts for
4 papers · 1 filter
Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectories
Zhehao Huang, Changxin Tian, Qingyuan Yang +5
Mid-training equips pretrained large language models with specialized and reasoning capabilities, but the returns of this stage are bounded since additional serial compute yields l…
Hyperparameter Scaling Laws Across MoE Sparsity
Changxin Tian, Kunlong Chen, Jia Liu +3
Mixture-of-Experts (MoE) models expand model capacity without a proportional increase in training compute, but increasing sparsity makes reliable hyperparameter transfer challengin…
MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Merging
Jiapeng Wang, Changxin Tian, Kunlong Chen +5
Optimizing data mixtures is essential for unlocking the full potential of large language models (LLMs), yet identifying the optimal composition remains computationally prohibitive…
POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications
Chunjing Gan, Dan Yang, Binbin Hu +5
Large language models (LLMs) have become a disruptive force in the industry, introducing unprecedented capabilities in natural language processing, logical reasoning and so on. How…