Desiderata for next generation of ML model serving

Sherif Akoush, Andrei Paleyes, Arnaud Van Looveren +1

Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This…