3 papers
cs.LG2025
A Faster Generalized Two-Stage Approximate Top-K
Yashas Samaga, Varun Yerram, Spandana Raj Babbula +2
We consider the Top- selection problem, which aims to identify the largest elements in an array. Top- selection arises in many machine learning algorithms and often becom…
cs.LG2024
HiRE: High Recall Approximate Top- Estimation for Efficient LLM Inference
Yashas Samaga B L, Varun Yerram, Chong You +4
Autoregressive decoding with generative Large Language Models (LLMs) on accelerators (GPUs/TPUs) is often memory-bound where most of the time is spent on transferring model paramet…
cs.AI2024
Tandem Transformers for Inference Efficient LLMs
Aishwarya P S, Pranav Ajit Nair, Yashas Samaga +4
The autoregressive nature of conventional large language models (LLMs) inherently limits inference speed, as tokens are generated sequentially. While speculative and parallel decod…