Making MoE-based LLM Inference Resilient with Tarragon

Songyu Zhang, Aaron Tam, Myungjin Lee +2

Mixture-of-Experts (MoE) models are increasingly used to serve LLMs at scale, but failures become common as deployment scale grows. Existing systems exhibit poor failure resilience…