most citedGatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models

1 citations · 1 across the 6 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2025

Virtual Width Networks

Seed, Baisheng Li, Banggu Wu +115

We introduce Virtual Width Networks (VWN), a framework that delivers the benefits of wider representations without incurring the quadratic cost of increasing the hidden size. VWN d…

cs.LG2025

UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning

Zihao Huang, Yu Bao, Qiyang Min +8

While Mixture of Experts (MoE) models achieve remarkable efficiency by activating only subsets of parameters, they suffer from high memory access costs during inference. Memory-lay…