3 citations · 5 across the 7 of their papers we have counts for
6 papers · 1 filter
FastKernels: Benchmarking GPU Kernel Generation in Production
Gabriele Oliaro, Yichao Fu, Jaeseong Lee +8
LLM-based agents for GPU kernel generation are advancing rapidly, but the benchmarks they optimize against evaluate kernels in isolation, with synthetic inputs and weak baselines,…
When Drafts Evolve: Speculative Decoding Meets Online Learning
Yu-Yang Qian, Hao-Cong Wu, Yichao Fu +2
Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens th…
Scaling Speculative Decoding with Lookahead Reasoning
Yichao Fu, Rui Ge, Zelei Shao +2
Reasoning models excel by generating long chain-of-thoughts, but decoding the resulting thousands of tokens is slow. Token-level speculative decoding (SD) helps, but its benefit is…
Efficiently Scaling LLM Reasoning with Certaindex
Yichao Fu, Junda Chen, Siqi Zhu +8
Test-time reasoning algorithms such as chain-of-thought, self-consistency, and MCTS enhance LLM problem-solving but can wastefully generate many tokens without improving accuracy.…
Efficient LLM Scheduling by Learning to Rank
Yichao Fu, Siqi Zhu, Runlong Su +3
In Large Language Model (LLM) inference, the output length of an LLM request is typically regarded as not known a priori. Consequently, most LLM serving systems employ a simple Fir…
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Yichao Fu, Peter Bailis, Ion Stoica +1
Autoregressive decoding of large language models (LLMs) is memory bandwidth bounded, resulting in high latency and significant wastes of the parallel processing power of modern acc…