26 citations · 33 across the 5 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Atindra Jha, Margaret Li, Jure Leskovec +2
As the supply of human-written text is exhausted, it has become standard practice to repeat language model training data. Prior work has studied data repetition for densely activat…
cs.LG2026
Slicing and Dicing: Configuring Optimal Mixtures of Experts
Margaret Li, Sneha Kudugunta, Danielle Rothermel +1
Mixture-of-Experts (MoE) architectures have become standard in large language models, yet many of their core design choices - expert count, granularity, shared experts, load balanc…