5 papers
ScaleAcross Explorer: Exploring Communication Optimization for Scale-Across AI Model Training
Minghao Li, Alicia Golden, Samuel Hsia +14
The rapid scaling of large language model training requires distributing GPU resources across multiple data center buildings and regions. We refer to such paradigm as "scale-across…
SVR-MAD: A Bayesian-Inspired Framework for Posterior-Guided Multi-Agent Debate
Weifan Jiang, Rana Shahout, Minghao Li +4
Multi-Agent Debate (MAD) improves LLM-agent accuracy but suffers from rapid context growth, limiting scalability in larger multi-agent settings. Existing methods prune low-utility…
PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
Can Hankendi, Rana Shahout, Minlan Yu +1
Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize…
Orla: A Library for Serving LLM-Based Multi-Agent Systems
Rana Shahout, Hayder Tirmazi, Minlan Yu +1
We introduce Orla, a library for constructing and running LLM-based agentic systems. Modern agentic applications consist of workflows that combine multiple LLM inference steps, too…
From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
Rana Shahout, Colin Cai, Yilun Du +2
Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces trainin…