From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs

Zhentao Tan, Chang Liu, Yao Liu +2

As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Expert…