8 papers
Confidence-Modulated Speculative Decoding for Large Language Models
Jaydip Sen, Subhasis Dasgupta, Hetvi Waghela
Speculative decoding has emerged as an effective approach for accelerating autoregressive inference by parallelizing token generation through a draft-then-verify paradigm. However,…
Predicting Road Crossing Behaviour using Pose Detection and Sequence Modelling
Subhasis Dasgupta, Preetam Saha, Agniva Roy +1
The world is constantly moving towards AI based systems and autonomous vehicles are now reality in different parts of the world. These vehicles require sensors and cameras to detec…
Hierarchical Verification of Speculative Beams for Accelerating LLM Inference
Jaydip Sen, Harshitha Puvvala, Subhasis Dasgupta
Large language models (LLMs) have achieved remarkable success across diverse natural language processing tasks but face persistent challenges in inference efficiency due to their a…
Determination Of Structural Cracks Using Deep Learning Frameworks
Subhasis Dasgupta, Jaydip Sen, Tuhina Halder
Structural crack detection is a critical task for public safety as it helps in preventing potential structural failures that could endanger lives. Manual detection by inexperienced…
Multi-Amateur Contrastive Decoding for Text Generation
Jaydip Sen, Subhasis Dasgupta, Hetvi Waghela
Contrastive Decoding (CD) has emerged as an effective inference-time strategy for enhancing open-ended text generation by exploiting the divergence in output probabilities between…
Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
Jaydip Sen, Saptarshi Sengupta, Subhasis Dasgupta
This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding…