Showing 2025 · cs.LGShow all
2 papers · 2 filters
cs.LG2025
Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization
Rahul Krishna Thomas, Arka Pal
Speculative sampling reduces the latency of autoregressive decoding for target model LLMs without sacrificing inference quality, by using a cheap draft model to suggest a candidate…
cs.LG2025
Cascade: Token-Sharded Private LLM Inference
Rahul Thomas, Louai Zahran, Erica Choi +3
As LLMs continue to increase in parameter size, the computational resources required to run them are available to fewer parties. Therefore, third-party inference services -- where…