7 papers
Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning
Tianyuan Shi, Canbin Huang, Bei Li +4
Distilling reasoning capabilities from strong to weak language models typically involves imitating specific solution trajectories, effectively transferring what to answer rather th…
When Model Merging Breaks Routing: Training-Free Calibration for MoE
Canbin Huang, Tianyuan Shi, Xiaojun Quan +3
Model merging has emerged as a cost-effective approach for consolidating the capabilities of multiple LLMs without retraining. However, existing merging techniques, largely based o…
Lookahead Routing for Large Language Models
Canbin Huang, Tianyuan Shi, Yuhua Zhu +2
Large language model (LLM) routers improve the efficiency of multi-model systems by directing each query to the most appropriate model while leveraging the diverse strengths of het…
Mutual-Taught for Co-adapting Policy and Reward Models
Tianyuan Shi, Canbin Huang, Fanqi Wan +5
During the preference optimization of large language models (LLMs), distribution shifts may arise between newly generated model samples and the data used to train the reward model…
FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
Ziyi Yang, Fanqi Wan, Longguang Zhong +3
We introduce FuseChat-3.0, a suite of large language models (LLMs) developed by integrating the strengths of heterogeneous source LLMs into more compact target LLMs. Our source mod…
ProFuser: Progressive Fusion of Large Language Models
Tianyuan Shi, Fanqi Wan, Canbin Huang +6
While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly…
Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System
Weizhou Shen, Yingqi Gao, Canbin Huang +3
Developing an efficient retriever to retrieve knowledge from a large-scale knowledge base (KB) is critical for task-oriented dialogue systems to effectively handle localized and sp…