#sparse mixture-of-experts

From Expert Reduction to Behavioral Divergence: Tracing Numerical State through Sparse MoE Inference

Tianyang Zhu

The paper studies how different expert reduction orders and numerical precision settings cause divergent behavior in sparse mixture-of-experts (MoE) inference, showing that identic…