54 citations · 55 across the 5 of their papers we have counts for
5 papers
FengHuang: Next-Generation Memory Orchestration for AI Inferencing
Jiamin Li, Lei Qu, Tao Zhang +4
This document presents a vision for a novel AI infrastructure design that has been initially validated through inference simulations on state-of-the-art large language models. Adva…
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
Bodun Hu, Jiamin Li, Le Xu +5
The increasing demand for Large Language Models (LLMs) across various applications has led to a significant shift in the design of deep learning serving systems. Deploying LLMs, pa…
Adaptive Gating in Mixture-of-Experts based Language Models
Jiamin Li, Qiang Su, Yitao Yang +3
Large language models, such as OpenAI's ChatGPT, have demonstrated exceptional language understanding capabilities in various NLP tasks. Sparsely activated mixture-of-experts (MoE)…
Accelerating Distributed MoE Training and Inference with Lina
Jiamin Li, Yimin Jiang, Yibo Zhu +2
Scaling model parameters improves model quality at the price of high computation overhead. Sparsely activated models, usually in the form of Mixture of Experts (MoE) architecture,…
Aryl: An Elastic Cluster Scheduler for Deep Learning
Jiamin Li, Hong Xu, Yibo Zhu +3
Companies build separate training and inference GPU clusters for deep learning, and use separate schedulers to manage them. This leads to problems for both training and inference:…