Sponge: Inference Serving with Dynamic SLOs Using In-Place Vertical Scaling

Kamran Razavi, Saeid Ghafouri, Max Mühlhäuser +2

Mobile and IoT applications increasingly adopt deep learning inference to provide intelligence. Inference requests are typically sent to a cloud infrastructure over a wireless netw…