1 citations · 1 across the 5 of their papers we have counts for
10 papers · 1 filter
The Art of Scaling Test-Time Compute for Large Language Models
Aradhye Agarwal, Ayan Sengupta, Tanmoy Chakraborty
Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a system…
Value-Guided KV Compression for LLMs via Approximated CUR Decomposition
Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
Key-value (KV) cache compression has emerged as a critical technique for reducing the memory and latency overhead of autoregressive language models during inference. Prior approach…
On the Generalization vs Fidelity Paradox in Knowledge Distillation
Suhas Kamasetty Ramesh, Ayan Sengupta, Tanmoy Chakraborty
Knowledge distillation (KD) is a key technique for compressing large language models into smaller ones while preserving performance. Despite the recent traction of KD research, its…
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
Ayan Sengupta, Vaibhav Seth, Arinjay Pathak +5
Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approac…
Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
Aradhye Agarwal, Suhas K Ramesh, Ayan Sengupta +1
Fine-tuning large language models (LLMs) on downstream tasks requires substantial computational resources. Selective PEFT, a class of parameter-efficient fine-tuning (PEFT) methodo…
How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines
Ayan Sengupta, Yash Goel, Tanmoy Chakraborty
Neural scaling laws have revolutionized the design and optimization of large-scale AI models by revealing predictable relationships between model size, dataset volume, and computat…