SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads

Alind Khare, Dhruv Garg, Sukrit Kalra +3

The increasing deployment of ML models on the critical path of production applications in both datacenter and the edge requires ML inference serving systems to serve these models u…