1 citations · 2 across the 7 of their papers we have counts for
7 papers
DEP: A Decentralized Large Language Model Evaluation Protocol
Jianxiang Peng, Junhao Li, Hongxiang Wang +15
With the rapid development of Large Language Models (LLMs), a large number of benchmarks have been proposed. However, most benchmarks lack unified evaluation standard and require t…
Make Every Draft Count: Hidden State based Speculative Decoding
Yuetao Chen, Xuliang Wang, Xinzhou Zheng +3
Speculative decoding has emerged as a pivotal technique to accelerate LLM inference by employing a lightweight draft model to generate candidate tokens that are subsequently verifi…
PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
Xuliang Wang, Yuetao Chen, Maochan Zhen +5
Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerat…
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training
Xin Tan, Yuetao Chen, Yimin Jiang +6
Diffusion Transformers (DiTs) have shown remarkable performance in generating high-quality videos. However, the quadratic complexity of 3D full attention remains a bottleneck in sc…
DeepServe: Serverless Large Language Model Serving at Scale
Junhao Hu, Jiang Xu, Zhixia Liu +18
In this paper, we propose DEEPSERVE, a scalable and serverless AI platform designed to efficiently serve large language models (LLMs) at scale in cloud environments. DEEPSERVE addr…
Echo: Simulating Distributed Training At Scale
Yicheng Feng, Yuetao Chen, Kaiwen Chen +7
Simulation offers unique values for both enumeration and extrapolation purposes, and is becoming increasingly important for managing the massive machine learning (ML) clusters and…