11 papers
ASymPO: Asymmetric-Scale Policy Optimization for Asynchronous LLM Post-Training Without Behavior Information
Zehua Liu, Yuxuan Yao, Xiaojin Fu +2
Asynchronous reinforcement learning can improve language-model post-training throughput by decoupling response generation from policy optimization, but stale responses introduce di…
Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
Yuxuan Yao, Haonan Sheng, Qingsong Lv +11
The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. Howev…
RIFT: Repurposing Negative Samples via Reward-Informed Fine-Tuning
Zehua Liu, Shuqi Liu, Tao Zhong +1
While Supervised Fine-Tuning (SFT) and Rejection Sampling Fine-Tuning (RFT) are standard for LLM alignment, they either rely on costly expert data or discard valuable negative samp…
REG: A Regularization Optimizer for Robust Training Dynamics
Zehua Liu, Han Wu, Xiaojin Fu +4
Optimizers are crucial for the efficient training of Large Language Models (LLMs). While AdamW is the de facto standard, recent structure-aware optimizers like Muon have emerged, w…
Activation-Guided Consensus Merging for Large Language Models
Yuxuan Yao, Shuqi Liu, Zehua Liu +6
Recent research has increasingly focused on reconciling the reasoning capabilities of System 2 with the efficiency of System 1. While existing training-based and prompt-based appro…
Automatic Operator-level Parallelism Planning for Distributed Deep Learning -- A Mixed-Integer Programming Approach
Ruifeng She, Bowen Pang, Kai Li +2
As the artificial intelligence community advances into the era of large models with billions of parameters, distributed training and inference have become essential. While various…
MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
Zehua Liu, Han Wu, Ruifeng She +4
Mixture of Experts (MoE) has become a key architectural paradigm for efficiently scaling Large Language Models (LLMs) by selectively activating a subset of parameters for each inpu…
Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
Han Wu, Yuxuan Yao, Shuqi Liu +7
The transition from System 1 to System 2 reasoning in large language models (LLMs) has marked significant advancements in handling complex tasks through deliberate, iterative think…
LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
Zehua Liu, Han Wu, Yuxuan Yao +4
While most current approaches rely on further training techniques, such as fine-tuning or reinforcement learning, to enhance model capacities, model merging stands out for its abil…
1bit-Merging: Dynamic Quantized Merging for Large Language Models
Shuqi Liu, Yuxuan Yao, Bowei He +5
Recent advances in large language models have led to specialized models excelling in specific domains, creating a need for efficient model merging techniques. While traditional mer…
BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving
Teng Wang, Wing-Yin Yu, Zhenqi He +8
LLMs exhibit advanced reasoning capabilities, offering the potential to transform natural language questions into mathematical models. However, existing open-source datasets in ope…