ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems

Mahdi Soleymani, Ramy E. Ali, Hessam Mahdavifar +1

Due to the surge of cloud-assisted AI services, the problem of designing resilient prediction serving systems that can effectively cope with stragglers/failures and minimize respon…