CascadeServe: Unlocking Model Cascades for Inference Serving

Ferdi Kossmann, Ziniu Wu, Alex Turk +3

Machine learning (ML) models are increasingly deployed to production, calling for efficient inference serving systems. Efficient inference serving is complicated by two challenges:…