Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness

Zeyuan Tan, Xiulong Yuan, Congjie He +7

Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number o…