Automated Runtime-Aware Scheduling for Multi-Tenant DNN Inference on GPU

Fuxun Yu, Shawn Bray, Di Wang +4

With the fast development of deep neural networks (DNNs), many real-world applications are adopting multiple models to conduct compound tasks, such as co-running classification, de…