Multi-user Co-inference with Batch Processing Capable Edge Server

Wenqi Shi, Sheng Zhou, Zhisheng Niu +2

Graphics processing units (GPUs) can improve deep neural network inference throughput via batch processing, where multiple tasks are concurrently processed. We focus on novel scena…