Mixtures of SubExperts for Large Language Continual Learning

Haeyong Kang, Hee Suk Yoon, Dahua Feng +1

Enabling lifelong learning in LLMs demands resolving the stability-plasticity dilemma (i.e., models must incorporate new knowledge without overwriting prior representations) while…