Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
WenHung Lee, Jian-Jia Chen, Xiaolin Lin +6
While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck whe…
cs.CL2025
Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
Pei-Shuo Wang, Jian-Jia Chen, Chun-Che Yang +4
The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strateg…