1 paper
Jingyuan Li, Maolin Wang, Hayden So
In expert-parallel Mixture-of-Experts serving, a layer finishes only when the most loaded GPU of its group does. Token routing shifts at every decode step and concentrates on a few…