InferLine: ML Prediction Pipeline Provisioning and Management for Tight Latency Objectives

Daniel Crankshaw, Gur-Eyal Sela, Corey Zumar +4

Serving ML prediction pipelines spanning multiple models and hardware accelerators is a key challenge in production machine learning. Optimally configuring these pipelines to meet…