4 papers
Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
Ao Xu, Han Zhao, Weihao Cui +7
Large language models (LLMs) are increasingly deployed under the Model-as-a-Service (MaaS) paradigm. To meet stringent quality-of-service (QoS) requirements, existing LLM serving s…
Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution
Shulai Zhang, Ao Xu, Quan Chen +6
Embodied AI systems operate in dynamic environments, requiring seamless integration of perception and generation modules to process high-frequency input and output demands. Traditi…
Towards Resource-Efficient Serverless LLM Inference with SLINFER
Chuhao Xu, Zijun Li, Quan Chen +3
The rise of LLMs has driven demand for private serverless deployments, characterized by moderate-sized models and infrequent requests. While existing serverless solutions follow ex…
Efficient Function-as-a-Service for Large Language Models with TIDAL
Weihao Cui, Ziyi Xu, Han Zhao +4
Large Language Model (LLM) applications have emerged as a prominent use case for Function-as-a-Service (FaaS) due to their high computational demands and sporadic invocation patter…