Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Distributionally Robust Mixture-of-Experts Training
Xin Teng, Muxiao Li, Hongyi Wen
Mixture-of-Experts (MoE) transformers scale capacity by activating only a few experts per token, but this sparsity creates a hidden reliability problem: when routing is imperfect,…
cs.LG2026
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
Xin Teng, Canyu Zhang, Shaoyi Zheng +3
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precomp…