Multi-model Machine Learning Inference Serving with GPU Spatial Partitioning

Seungbeom Choi, Sunho Lee, Yeonjae Kim +3

As machine learning techniques are applied to a widening range of applications, high throughput machine learning (ML) inference servers have become critical for online service appl…