24 citations · 34 across the 9 of their papers we have counts for
1 paper · 2 filters
Simon Schug
Sparsity allows scaling model parameters without proportionally increasing computational cost. While mixture of experts (MoE) models are made increasingly sparse, individual expert…