PM2Lat: Highly Accurate and Generalized Prediction of DNN Execution Latency on GPUs

Truong-Thanh Le, Hoang-Loc La, Amir Taherkordi +3

We present PM2Lat, a fast and generalized framework for accurately predicting the latency of deep neural network models on GPUs, with special focus on NVIDIA. Unlike prior methods…