activity
20222026
most citedAlexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

64 citations · 104 across the 6 of their papers we have counts for

collaborators
Showing 2026Show all

1 paper · 1 filter

cs.LG2026

MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale

Tobias Falke, Nicolas Anastassacos, Samson Tan +6

Sparse Mixture-of-Experts (MoE) architectures are increasingly popular for frontier large language models (LLM) but they introduce training challenges due to routing complexity. Fu…