6 citations · 11 across the 15 of their papers we have counts for
4 papers · 1 filter
Shared LoRA Subspaces for almost Strict Continual Learning
Prakhar Kaushik, Ankit Vaidya, Shravan Chaudhari +2
Adapting large pretrained models to new tasks efficiently and continually is crucial for real-world deployment but remains challenging due to catastrophic forgetting and the high c…
The Universal Weight Subspace Hypothesis
Prakhar Kaushik, Shravan Chaudhari, Ankit Vaidya +2
We show that deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces. We provide the first large-scale empirical evidence…
EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference
Prakhar Kaushik, Ankit Vaidya, Shravan Chaudhari +1
The rapid growth of large models has raised concerns about their environmental impact and equity in accessibility due to significant computational costs. Low-Rank Adapters (LoRA) o…
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Prakhar Kaushik, Alex Gain, Adam Kortylewski +1
Catastrophic forgetting in neural networks is a significant problem for continual learning. A majority of the current methods replay previous data during training, which violates t…