Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning

Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han +8

Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. E…