A Survey of Multi-Tenant Deep Learning Inference on GPU

Fuxun Yu, Di Wang, Longfei Shangguan +3

Deep Learning (DL) models have achieved superior performance. Meanwhile, computing hardware like NVIDIA GPUs also demonstrated strong computing scaling trends with 2x throughput an…