8 papers
Revisiting Model Interpolation for Efficient Reasoning
Taiqiang Wu, Runming Yang, Tao Liu +2
Model merging, typically on Instruct and Thinking models, has shown remarkable performance for efficient reasoning. In this paper, we systematically revisit the simplest merging me…
Timber: Training-free Instruct Model Refining with Base via Effective Rank
Taiqiang Wu, Runming Yang, Tao Liu +3
Post-training, which elicits a pretrained Base model into the corresponding Instruct model, is widely considered to be superficial. In this work, we first reinforce this hypothesis…
PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
He Xiao, Runming Yang, Qingyao Yang +6
Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency a…
Shadow-FT: Tuning Instruct Model via Training on Paired Base Model
Taiqiang Wu, Runming Yang, Jiayi Li +4
Large language models (LLMs) consistently benefit from further fine-tuning on various tasks. However, we observe that directly tuning the Instruct (i.e., instruction-tuned) models…
Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning
Zhen Li, Yupeng Su, Songmiao Wang +8
Low-bit post-training quantization (PTQ) is a practical route to deploy reasoning-capable LLMs under tight memory and latency budgets, yet it can markedly impair mathematical reaso…
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
Zhen Li, Yupeng Su, Runming Yang +5
Large language models have achieved significant advancements in complex mathematical reasoning benchmarks, such as MATH. However, their substantial computational requirements prese…