activity
20222026
most citedSpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification

101 citations · 162 across the 6 of their papers we have counts for

collaborators
Showing 2024Show all

2 papers · 1 filter

cs.LG2024★ 1 cited

Mirage: A Multi-Level Superoptimizer for Tensor Programs

Mengdi Wu, Xinhao Cheng, Shengyu Liu +7

We introduce Mirage, the first multi-level superoptimizer for tensor programs. A key idea in Mirage is Graphs, a uniform representation of tensor programs at the kernel, thread…

cs.CL2024

Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge

Bin Xiao, Chunan Shi, Xiaonan Nie +5

Large language models (LLMs) suffer from low efficiency as the mismatch between the requirement of auto-regressive decoding and the design of most contemporary GPUs. Specifically,…