1 citations · 1 across the 3 of their papers we have counts for
1 paper · 1 filter
Leyla Mirvakhabova, Babak Ehteshami Bejnordi, Gaurav Kumar +3
Upcycling pre-trained dense models into sparse Mixture-of-Experts (MoEs) efficiently increases model capacity but often suffers from poor expert specialization due to naive weight…