Efficient Training of Diffusion Mixture-of-Experts Models: A Practical Recipe

Yahui Liu, Yang Yue, Jingyuan Zhang +5

Recent efforts on Diffusion Mixture-of-Experts (MoE) models have primarily focused on developing more sophisticated routing mechanisms. However, we observe that the underlying arch…