MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting

Tianhao Li, Shangjie Li, Binbin Xie +2

The advent of large language models (LLMs) has predominantly catered to high-resource languages, leaving a disparity in performance for low-resource languages. Conventional Continu…