6 citations · 10 across the 6 of their papers we have counts for
7 papers
Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference
Jianxing Qin, Alexander Du, Danfeng Zhang +2
LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect s…
Hydra: Efficient, Correct Code Generation via Checkpoint-and-Rollback Support
Alexander Du, Jianjun Ou, Danyang Zhuo +1
Large language models are increasingly used for code generation, but many generated programs fail to compile, a prerequisite for further correctness checks such as unit tests. Exis…
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
Yongji Wu, Xueshen Liu, Shuowei Jin +6
The Mixture-of-Experts (MoE) architecture has become increasingly popular as a method to scale up large language models (LLMs). To save costs, heterogeneity-aware training solution…
Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models
Yongji Wu, Wenjie Qu, Xueshen Liu +10
Sparsely-activated Mixture-of-Experts (MoE) architecture has increasingly been adopted to further scale large language models (LLMs). However, frequent failures still pose signific…
VcLLM: Video Codecs are Secretly Tensor Codecs
Ceyu Xu, Yongji Wu, Xinyu Yang +4
As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottleneck…
Plato: Plan to Efficiently Decode for Large Language Model Inference
Shuowei Jin, Xueshen Liu, Yongji Wu +7
Large language models (LLMs) have achieved remarkable success in natural language tasks, but their inference incurs substantial computational and memory overhead. To improve effici…
Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native
Yao Lu, Song Bian, Lequn Chen +19
In this paper, we investigate the intersection of large generative AI models and cloud-native computing architectures. Recent large models such as ChatGPT, while revolutionary in t…