collaborators

1 paper

cs.CL2024

Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules

Zhuocheng Gong, Ang Lv, Jian Guan +6

Is it always necessary to compute tokens from shallow to deep layers in Transformers? The continued success of vanilla Transformers and their variants suggests an undoubted "yes".…