TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

Guanzhi Deng, Haibo Wang, Kuan Wu +5

Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream…