363 citations · 496 across the 25 of their papers we have counts for
4 papers · 1 filter
Lodestar: An Online-Learning LLM Inference Router
Gangmuk Lim, Wanyu Zhao, Brighten Godfrey +3
Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for GPU utilization. However, LLM r…
Kivi: Verification for Cluster Management
Bingzhe Liu, Gangmuk Lim, Ryan Beckett +1
Modern cloud infrastructure is powered by cluster management systems such as Kubernetes and Docker Swarm. While these systems seek to minimize users' operational burden, the comple…
Shortest Paths in Microseconds
Rachit Agarwal, Matthew Caesar, P. Brighten Godfrey +1
Computing shortest paths is a fundamental primitive for several social network applications including socially-sensitive ranking, location-aware search, social auctions and social…
On the Resilience of Routing Tables
Joan Feigenbaum, Brighten Godfrey, Aurojit Panda +3
Many modern network designs incorporate "failover" paths into routers' forwarding tables. We initiate the theoretical study of the conditions under which such resilient routing tab…