2 citations · 5 across the 13 of their papers we have counts for
5 papers · 1 filter
A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
Yue Zhang, Yuansheng Chen, Xuan Mo +3
LLM inference serving typically scales out with a two-tier architecture: a cluster router distributes requests to multiple inference engines, each of which then in turn performs it…
MaaSO: SLO-aware Orchestration of Heterogeneous Model Instances for MaaS
Mo Xuan, Zhang yue, Wu Weigang
Model-as-a-Service (MaaS) platforms face diverse Service Level Objective (SLO) requirements stemming from various large language model (LLM) applications, manifested in contextual…
Chain of Methodologies: Scaling Test Time Computation without Training
Cong Liu, Jie Wu, Weigang Wu +3
Large Language Models (LLMs) often struggle with complex reasoning tasks due to insufficient in-depth insights in their training data, which are typically absent in publicly availa…
Analysis of Asynchronous Federated Learning: Unraveling the Interactions between Gradient Compression, Delay, and Data Heterogeneity
Diying Yang, Yingwei Hou, Weigang Wu
In practical federated learning (FL), the large communication overhead between clients and the server is often a significant bottleneck. Gradient compression methods can effectivel…
Grey-Box Fuzzing in Constrained Ultra-Large Systems: Lessons for SE Community
Jiazhao Yu, Yanlun Tu, Zhanlei Zhang +5
Testing ultra-large microservices-based FinTech systems presents significant challenges, including restricted access to production environments, complex dependencies, and stringent…