2 citations · 2 across the 7 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Sparse Mixture-of-Experts for Compositional Generalization: Empirical Evidence and Theoretical Foundations of Optimal Sparsity
Jinze Zhao, Peihao Wang, Junjie Yang +6
Sparse Mixture-of-Experts (SMoE) architectures have gained prominence for their ability to scale neural networks, particularly transformers, without a proportional increase in comp…
cs.LG2024★ 2 cited
Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study
Jinze Zhao, Peihao Wang, Zhangyang Wang
Mixture-of-Experts (MoE) represents an ensemble methodology that amalgamates predictions from several specialized sub-models (referred to as experts). This fusion is accomplished t…