9 citations · 9 across the 4 of their papers we have counts for
1 paper · 1 filter
Bowen Dong, Yilong Fan, Tengyu Pan +6
Mixture-of-Depths (MoD) enables conditional computation across Transformer depth by routing only a subset of tokens through selected layers, but its original one-sparse--one-dense…