Packrat: Automatic Reconfiguration for Latency Minimization in CPU-based DNN Serving

Ankit Bhardwaj, Amar Phanishayee, Deepak Narayanan +2

In this paper, we investigate how to push the performance limits of serving Deep Neural Network (DNN) models on CPU-based servers. Specifically, we observe that while intra-operato…