5 papers
What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
Keyu Lv, Manyi Zhang, Xiaobo Xia +6
Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training…
E-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
Tao Yuan, Haoli Bai, Yinfei Pan +5
With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruni…
A Simple Linear Patch Revives Layer-Pruned Large Language Models
Xinrui Chen, Haoli Bai, Tao Yuan +7
Layer pruning has emerged as a widely used technique for compressing large language models (LLMs). However, existing layer pruning approaches often incur substantial performance de…
Faster and Better LLMs via Latency-Aware Test-Time Scaling
Zili Wang, Tianyu Zhang, Haoli Bai +5
Test-Time Scaling (TTS) has proven effective in improving the performance of Large Language Models (LLMs) during inference. However, existing research has overlooked the efficiency…
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
Ruikang Liu, Yuxuan Sun, Manyi Zhang +5
Recent advancements in reasoning language models have demonstrated remarkable performance in complex tasks, but their extended chain-of-thought reasoning process increases inferenc…