172 citations · 257 across the 17 of their papers we have counts for
19 papers
Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
Arnav Chavan, Kusha Sareen, Rishabh Tiwari +6
Large language models (LLMs) acquire new skills through fine-tuning, but it can destroy capabilities they already have: catastrophic forgetting remains a central obstacle to contin…
Jacobian-guided Noise Injection for Quantization Robustness in Large Language Models
Deepanshu Pandey, Arnav Chavan, Nahush Lele +2
Quantization of Large Language Models (LLMs) is often hindered by the sensitivity of the self-attention mechanism to discretization errors. We identify the softmax operator as a bo…
DOT-MoE: Differentiable Optimal Transport for MoEfication
Udbhav Bamba, Arnav Chavan, Aryamaan Thakur +2
The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency. While Mixture of Experts (MoEs) arc…
S2D: Selective Spectral Decay for Quantization-Friendly Conditioning of Neural Activations
Arnav Chavan, Nahush Lele, Udbhav Bamba +3
Activation outliers in large-scale transformer models pose a fundamental challenge to model quantization, creating excessively large ranges that cause severe accuracy drops during…
Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion
Koustav Ghosal, Abhranta Panigrahi, Arnav Chavan +2
Seismic full waveform inversion (FWI) has seen promising advancements through deep learning. Existing approaches typically focus on task-specific models trained and evaluated in is…
Surgical Feature-Space Decomposition of LLMs: Why, When and How?
Arnav Chavan, Nahush Lele, Deepak Gupta
Low-rank approximations, of the weight and feature space can enhance the performance of deep learning models, whether in terms of improving generalization or reducing the latency o…