2 papers
cs.LG2026
Exact Quantile Balancing and Load-Error Injection for Mixture-of-Experts
Pit Neitemeier, Jiaze Li, Alessio Serra +2
Mixture-of-Experts (MoE) training requires global load balance to prevent expert under-utilization and local balance for efficient expert-parallel execution. Existing distributed Q…
cs.LG2026
SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence Models
Pit Neitemeier, Alessio Serra, Jiaze Li +3
Hierarchical sequence models replace fixed tokenization with learned segmentations that compress long byte sequences for efficient autoregressive modeling. While recent end-to-end…