25 citations · 40 across the 5 of their papers we have counts for
5 papers
ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving
Xiangchen Li, Saeid Ghafouri, Jiakun Fan +3
Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from heavyweight verification. While…
Sponge: Inference Serving with Dynamic SLOs Using In-Place Vertical Scaling
Kamran Razavi, Saeid Ghafouri, Max Mühlhäuser +2
Mobile and IoT applications increasingly adopt deep learning inference to provide intelligence. Inference requests are typically sent to a cloud infrastructure over a wireless netw…
HEET: A Heterogeneity Measure to Quantify the Difference across Distributed Computing Systems
Ali Mokhtari, Saeid Ghafouri, Pooyan Jamshidi +1
Although system heterogeneity has been extensively studied in the past, there is yet to be a study on measuring the impact of heterogeneity on system performance. For this purpose,…
IPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiency
Saeid Ghafouri, Kamran Razavi, Mehran Salmani +5
Efficiently optimizing multi-model inference pipelines for fast, accurate, and cost-effective inference is a crucial challenge in machine learning production systems, given their t…
Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems
Mehran Salmani, Saeid Ghafouri, Alireza Sanaee +5
The use of machine learning (ML) inference for various applications is growing drastically. ML inference services engage with users directly, requiring fast and accurate responses.…