FailLite: Failure-Resilient Model Serving for Resource-Constrained Edge Environments

Li Wu, Walid A. Hanafy, Tarek Abdelzaher +3

Model serving systems have become popular for deploying deep learning models for various latency-sensitive inference tasks. While traditional replication-based methods have been us…