13 citations · 17 across the 12 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024★ 2 cited
ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
Ziteng Wang, Jun Zhu, Jianfei Chen
Sparsely activated Mixture-of-Experts (MoE) models are widely adopted to scale up model capacity without increasing the computation budget. However, vanilla TopK routers are traine…
cs.LG2024
Efficient Backpropagation with Variance-Controlled Adaptive Sampling
Ziteng Wang, Jianfei Chen, Jun Zhu
Sampling-based algorithms, which eliminate ''unimportant'' computations during forward and/or back propagation (BP), offer potential solutions to accelerate neural network training…