4 papers · 1 filter
Mitigating Forgetting in Continual Learning with Selective Gradient Projection
Anika Singh, Aayush Dhaulakhandi, Varun Chopade +3
As neural networks are increasingly deployed in dynamic environments, they face the challenge of catastrophic forgetting, the tendency to overwrite previously learned knowledge whe…
A Few Bad Neurons: Isolating and Surgically Correcting Sycophancy
Claire O'Brien, Jessica Seto, Dristi Roy +6
Behavioral alignment in large language models (LLMs) is often achieved through broad fine-tuning, which can result in undesired side effects like distributional shift and low inter…
Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits
Dev Patel, Gabrielle Gervacio, Diekola Raimi +5
Large Language Models require substantial computational resources for inference, posing deployment challenges. While dynamic pruning offers superior efficiency over static methods…
Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
Hao Gu, Vibhas Nair, Amrithaa Ashok Kumar +2
Interpreting language models often involves circuit analysis, which aims to identify sparse subnetworks, or circuits, that accomplish specific tasks. Existing circuit discovery alg…