PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning

Min Jae Jung, JooHee Kim

Large Language Models (LLMs) encounter significant challenges in continual learning due to catastrophic forgetting, where new information overwrites previously acquired knowledge.…