AntBatchInfer: Elastic Batch Inference in the Kubernetes Cluster

Siyuan Li, Youshao Xiao, Fanzhuang Meng +4

Offline batch inference is a common task in the industry for deep learning applications, but it can be challenging to ensure stability and performance when dealing with large amoun…