Does Role Specialization Matter for Explanation Faithfulness in Mixture-of-Experts?

Yeji Kim, Housam Babiker, Mi-Young Kim +1

Mixture-of-Experts (MoE) architectures have recently been extended with role-based mechanisms for interpretability. This is typically done by assigning semantic roles to individual…