KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling

Guilin Zhang, Wulan Guo, Ziqi Tan +2

Autoscaling GPU inference workloads in Kubernetes remains challenging due to the reactive and threshold-based nature of default mechanisms such as the Horizontal Pod Autoscaler (HP…