Optimizing Prediction Serving on Low-Latency Serverless Dataflow

Vikram Sreekanti, Harikaran Subbaraj, Chenggang Wu +2

Prediction serving systems are designed to provide large volumes of low-latency inferences machine learning models. These systems mix data processing and computationally intensive…