On the Cost of Model-Serving Frameworks: An Experimental Evaluation

Pasquale De Rosa, Yérom-David Bromberg, Pascal Felber +2

In machine learning (ML), the inference phase is the process of applying pre-trained models to new, unseen data with the objective of making predictions. During the inference phase…