4 papers · 1 filter
FMplex: Model Virtualization for Serving Extensible Foundation Models
Hetvi Shastri, Pragya Sharma, Walid A. Hanafy +3
Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving system…
Collaborative Processing for Multi-Tenant Inference on Memory-Constrained Edge TPUs
Nathan Ng, Walid A. Hanafy, Prashanthi Kadambi +5
IoT applications increasingly rely on on-device AI accelerators to ensure high performance, especially in low-connectivity and safety-critical scenarios. However, the limited on-ch…
CarbonEdge: Leveraging Mesoscale Spatial Carbon-Intensity Variations for Low Carbon Edge Computing
Li Wu, Walid A. Hanafy, Abel Souza +5
The proliferation of latency-critical and compute-intensive edge applications is driving increases in computing demand and carbon emissions at the edge. To better understand carbon…
FailLite: Failure-Resilient Model Serving for Resource-Constrained Edge Environments
Li Wu, Walid A. Hanafy, Tarek Abdelzaher +3
Model serving systems have become popular for deploying deep learning models for various latency-sensitive inference tasks. While traditional replication-based methods have been us…