17 citations · 19 across the 11 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
cs.CL2026
Kolibri: A Sovereign European Model on the Pareto Frontier
Aleph Alpha, :, Moritz Althaus +65
This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under t…
cs.LG2026
Exact Quantile Balancing and Load-Error Injection for Mixture-of-Experts
Pit Neitemeier, Jiaze Li, Alessio Serra +2
Mixture-of-Experts (MoE) training requires global load balance to prevent expert under-utilization and local balance for efficient expert-parallel execution. Existing distributed Q…
cs.AI2026
Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack
Sohir Maskey, Philipp Scholl, Jonas Knupp +2
Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the best starting point for subse…