1 citations · 1 across the 11 of their papers we have counts for
5 papers · 1 filter
Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
Jeanmely Rojas Nunez, Viraj Sawant, Nathan Allen +4
Fine-tuning large language models (LLMs) frequently induces catastrophic forgetting of prior capabilities. Recent work has shown that reinforcement learning (RL) retains prior capa…
Mitigating Forgetting in Continual Learning with Selective Gradient Projection
Anika Singh, Aayush Dhaulakhandi, Varun Chopade +3
As neural networks are increasingly deployed in dynamic environments, they face the challenge of catastrophic forgetting, the tendency to overwrite previously learned knowledge whe…
Peek-a-Boo Reasoning: Contrastive Region Masking in MLLMs
Isha Chaturvedi, Anjana Nair, Yushen Li +5
We introduce Contrastive Region Masking (CRM), a training free diagnostic that reveals how multimodal large language models (MLLMs) depend on specific visual regions at each step o…
Universal Neurons in GPT-2: Emergence, Persistence, and Functional Impact
Advey Nandan, Cheng-Ting Chou, Amrit Kurakula +4
We investigate the phenomenon of neuron universality in independently trained GPT-2 Small models, examining these universal neurons-neurons with consistently correlated activations…
From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs
Stanley Yu, Vaidehi Bulusu, Oscar Yasunaga +5
Large Language Models (LLMs) exhibit strong conversational abilities but often generate falsehoods. Prior work suggests that the truthfulness of simple propositions can be represen…