14 papers
Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization
Palaash Goel, Ayan Sengupta, Akshay Nambi +1
Structured pruning is a promising approach for compressing large language models (LLMs), yet existing methods rely heavily on greedy heuristics that produce myopic decisions, and o…
Latent Performance Profiling of Large Language Models
Tanmoy Chakraborty, Ayan Sengupta, Suparna Bhattacharya +7
Large language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of their capabilities. Evaluating open-source…
From Images to Words: Efficient Cross-Modal Knowledge Distillation to Language Models from Black-box Teachers
Ayan Sengupta, Shantanu Dixit, Md Shad Akhtar +1
Knowledge distillation (KD) methods are pivotal in compressing large pre-trained language models into smaller models, ensuring computational efficiency without significantly droppi…
Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics
Samhruth Ananthanarayanan, Ayan Sengupta, Tanmoy Chakraborty
As context windows in LLMs scale to 100K+ tokens, the key-value (KV) cache becomes the dominant memory bottleneck, with recent methods claiming 80-90% savings and minimal benchmark…
The Art of Scaling Test-Time Compute for Large Language Models
Aradhye Agarwal, Ayan Sengupta, Tanmoy Chakraborty
Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a system…
Value-Guided KV Compression for LLMs via Approximated CUR Decomposition
Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
Key-value (KV) cache compression has emerged as a critical technique for reducing the memory and latency overhead of autoregressive language models during inference. Prior approach…