6 papers
Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows
Bochao Feng, Jianjiang Li, Haojie Wang +4
Agentic LLM workflows consist of sequences of model turns interleaved with tool interactions, so their end-to-end completion time depends not only on inference speed but also on wh…
Epoch: Compiling Diffusion Blocks for Sparse MoE Serving
Jianian Zhu, Hang Wu, Yinghui Li +3
Diffusion language models generate text by refining a fixed-size block of token positions through many forward passes, a loop that does not match the per-forward execution unit use…
Toward Native Multimodal Modeling: A Roadmap
Siyu An, Junru Lu, Junnan Dong +18
Multimodal modeling represents a vital step from modality-agnostic reasoning toward world modeling. While early approaches predominantly rely on late-fusion that assembles encoders…
NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
Jiacheng Huang, Zimin Li, Yinghui Li +1
The emergence of deep learning domain-specific languages (DSLs) has substantially reduced the obstacles in developing high-performance, cross-platform compute kernels. However, cur…
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
Hang Wu, Jianian Zhu, Yinghui Li +3
Large Language Models (LLMs) present a critical trade-off between inference quality and computational cost: larger models offer superior capabilities but incur significant latency,…
FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
Jianian Zhu, Hang Wu, Haojie Wang +4
Multi-modal Large Language Models (MLLMs) serving systems commonly employ KV-cache compression to reduce memory footprint. However, existing compression methods introduce significa…