101 citations · 150 across the 9 of their papers we have counts for
10 papers
Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel
Hongyi Jin, Bohan Hou, Guanjie Wang +18
Modern GPU workloads, especially large language model (LLM) inference, suffer from kernel launch overheads and coarse synchronization that limit inter-kernel parallelism. Recent me…
MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
Xinhao Cheng, Zhihao Zhang, Yu Zhou +17
We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kerne…
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Rui Pan, Yinwei Dai, Zhihao Zhang +3
Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs).…
Communication Bounds for the Distributed Experts Problem
Zhihao Jia, Qi Pang, Trung Tran +3
In this work, we study the experts problem in the distributed setting where an expert's cost needs to be aggregated across multiple servers. Our study considers various communicati…
AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
Zikun Li, Zhuofu Chen, Remi Delacourt +11
Modern large language model (LLM) applications exhibit diverse service-level objectives (SLOs), from low-latency requirements in interactive coding assistants to more relaxed const…
TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
Lijie Yang, Zhihao Zhang, Zhuofu Chen +2
Large language models (LLMs) have driven significant advancements across diverse NLP tasks, with long-context models gaining prominence for handling extended inputs. However, the e…