4 papers
Coordinated Scheduling for MoE LLM Serving
Yifan Sun, Zhexiang Zhang, Jiantong Jiang +5
Serving Mixture-of-Experts (MoE) large language models (LLMs) is challenging because dynamic request workloads interact with sparse expert routing, creating both data-parallel (DP)…
Janus: Disaggregating Attention and Experts for Scalable MoE Inference
Zhexiang Zhang, Ye Wang, Yumiao Zhao +10
Serving large Mixture-of-Experts (MoE) models is challenging because of their large memory footprints, heterogeneous resource demands, and highly dynamic inference workloads. Most…
DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment
Liang Zhu, Feiteng Fang, Yuelin Bai +4
Reinforcement Learning from Human Feedback (RLHF), using algorithms like Proximal Policy Optimization (PPO), aligns Large Language Models (LLMs) with human values but is costly and…
Online Marketplace: A Benchmark for Data Management in Microservices
Rodrigo Laigner, Zhexiang Zhang, Yijian Liu +2
Microservice architectures have become a popular approach for designing scalable distributed applications. Despite their extensive use in industrial settings for over a decade, the…