4 papers
Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling
Xinglin Wang, Jiayi Shi, Shaoxiong Feng +8
Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However,…
PatternKV: Flattening KV Representation Expands Quantization Headroom
Ji Zhang, Yiwei Li, Shaoxiong Feng +8
KV cache in autoregressive LLMs eliminates redundant recomputation but has emerged as the dominant memory and bandwidth bottleneck during inference, notably with long contexts and…
Speculative Decoding for Multi-Sample Inference
Yiwei Li, Jiayi Shi, Shaoxiong Feng +8
We propose a novel speculative decoding method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling. Our method exploits the intrinsic con…
Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation
Yiwei Li, Ji Zhang, Shaoxiong Feng +8
Self-consistency improves reasoning by aggregating diverse stochastic samples, yet the dynamics behind its efficacy remain underexplored. We reframe self-consistency as a dynamic d…