10 citations · 16 across the 8 of their papers we have counts for
8 papers
Janus: Disaggregating Attention and Experts for Scalable MoE Inference
Zhexiang Zhang, Ye Wang, Yumiao Zhao +9
Serving large Mixture-of-Experts (MoE) models is challenging because of their large memory footprints, heterogeneous resource demands, and highly dynamic inference workloads. Most…
Making Serverless Computing Extensible: A Case Study of Serverless Data Analytics
Minchen Yu, Yinghao Ren, Jiamu Zhao +1
Serverless computing has attracted a broad range of applications due to its ease of use and resource elasticity. However, developing serverless applications often poses a dilemma -…
ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
Kaiwen Chen, Xin Tan, Minchen Yu +2
Large Reasoning Models (LRMs) are becoming integral to many AI inference systems, enhancing their capabilities with advanced reasoning. However, deploying these models in productio…
λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
Minchen Yu, Rui Yang, Chaobo Jia +9
Serverless computing has emerged as a compelling solution for cloud-based model inference. However, as modern large language models (LLMs) continue to grow in size, existing server…
FaaSTube: Optimizing GPU-oriented Data Transfer for Serverless Computing
Hao Wu, Junxiao Deng, Minchen Yu +5
Serverless computing has gained significant traction for machine learning inference applications, which are often deployed as serverless workflows consisting of multiple CPU and GP…
CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
Suyi Li, Hanfeng Lu, Tianyuan Wu +6
Pre-trained large language models (LLMs) often need specialization for domain-specific tasks. Low-Rank Adaptation (LoRA) is a popular approach that adapts a base model to multiple…
Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference
Minchen Yu, Ao Wang, Dong Chen +8
Serverless computing offers a compelling cloud model for online inference services. However, existing serverless platforms lack efficient support for GPUs, hindering their ability…
Following the Data, Not the Function: Rethinking Function Orchestration in Serverless Computing
Minchen Yu, Tingjia Cao, Wei Wang +1
Serverless applications are typically composed of function workflows in which multiple short-lived functions are triggered to exchange data in response to events or state changes.…