A House United Within Itself: SLO-Awareness for On-Premises Containerized ML Inference Clusters via Faro

Beomyeol Jeon, Chen Wang, Diana Arroyo +2

This paper tackles the challenge of running multiple ML inference jobs (models) under time-varying workloads, on a constrained on-premises production cluster. Our system Faro takes…