5 papers
PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
Nan Jiang, Panagiotis Theodoropoulos, Wei Deng
Power sampling improves large language model (LLM) reasoning at test time by sampling from a sharpened version of the base model's distribution. Its sampler, PowerMH, is slow becau…
Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling
Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan +4
Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models (LLMs). High-probability seq…
DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification
Ziyi Wang, Siva Rajesh Kasa, Ankith M S +6
Speculative decoding is an effective technique for accelerating large language model inference by drafting multiple tokens in parallel. In practice, its speedup is often bottleneck…
SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
Ziyi Wang, Nan Jiang, Guang Lin +1
Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight pruning or low-bit quantization indivi…
Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong +5
Fast and high-quality language generation is the holy grail that people pursue in the age of AI. In this work, we introduce Discrete Diffusion Divergence Instruct (DiDi-Instruct),…