Towards Resiliency in Large Language Model Serving with KevlarFlow

Shangshu Qian, Kipling Liu, P. C. Sruthi +2

Large Language Model (LLM) serving systems remain fundamentally fragile, where frequent hardware faults in hyperscale clusters trigger disproportionate service outages in the softw…