3 citations · 3 across the 6 of their papers we have counts for
7 papers
Coda: Exploiting Admission Flexibility for Coding-Agent Serving
Youhe Jiang, Fangcheng Fu, Binhang Yuan +5
Coding agents powered by large language models (LLMs) repeatedly alternate between model inference and tool calls, creating long-lived sessions with reusable key-value (KV) states…
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7
As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through sp…
Provisioning to Runtime Optimization of a 100 MW-Scale AI Cluster
Ehsan K. Ardestani, Leonardo Piga, Jovan Stojkovic +19
The electric power supply for AI data centers is now the most significant bottleneck in the race toward Artificial General Intelligence, surpassing even the constraint of AI accele…
MTrainS: Improving DLRM training efficiency using heterogeneous memories
Hiwot Tadese Kassa, Paul Johnson, Jason Akers +7
Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires…
Building a Performance Model for Deep Learning Recommendation Model Training on GPUs
Zhongyi Lin, Louis Feng, Ehsan K. Ardestani +5
We devise a performance model for GPU training of Deep Learning Recommendation Models (DLRM), whose GPU utilization is low compared to other well-optimized CV and NLP models. We sh…
Supporting Massive DLRM Inference Through Software Defined Memory
Ehsan K. Ardestani, Changkyu Kim, Seung Jae Lee +17
Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1.5x per year. With model size soon to be in tera…
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang +50
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand…