8 citations · 8 across the 1 of their papers we have counts for
1 paper · 1 filter
Costin-Andrei Oncescu, Qingyang Wu, Wai Tong Chung +5
An increasing number of LLMs employ Mixture-of-Experts (MoE) architectures where the feed-forward layer is replaced by a pool of experts and each token only activates a small subse…