9 citations · 14 across the 12 of their papers we have counts for
3 papers · 2 filters
Confidence-Modulated Speculative Decoding for Large Language Models
Jaydip Sen, Subhasis Dasgupta, Hetvi Waghela
Speculative decoding has emerged as an effective approach for accelerating autoregressive inference by parallelizing token generation through a draft-then-verify paradigm. However,…
Hierarchical Verification of Speculative Beams for Accelerating LLM Inference
Jaydip Sen, Harshitha Puvvala, Subhasis Dasgupta
Large language models (LLMs) have achieved remarkable success across diverse natural language processing tasks but face persistent challenges in inference efficiency due to their a…
Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
Jaydip Sen, Saptarshi Sengupta, Subhasis Dasgupta
This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding…