2 citations · 3 across the 9 of their papers we have counts for
9 papers
Epoch: Compiling Diffusion Blocks for Sparse MoE Serving
Jianian Zhu, Hang Wu, Yinghui Li +3
Diffusion language models generate text by refining a fixed-size block of token positions through many forward passes, a loop that does not match the per-forward execution unit use…
Odin: Primitive-Level Synchronization for Distributed Point-Based Neural Rendering
Zhenxiang Ma, Zeyu He, Yuanzhen Zhou +6
Point-based neural rendering (PBNR) represents 3D scenes as explicit, trainable primitives and underpins high-quality reconstruction and emerging embodied AI and world-model pipeli…
DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse
Yuyang Chen, Runxin Zhong, Zan Zong +3
Recent advances in AI-generated content have driven widespread adoption of Diffusion Transformers (DiTs) for high-resolution, long-duration content generation. While parallelizatio…
FlashFuser: Expanding the Scale of Kernel Fusion for Compute-Intensive Operators via Inter-Core Connection
Ziyu Huang, Yangjie Zhou, Zihan Liu +8
The scaling of computation throughput continues to outpace improvements in memory bandwidth, making many deep learning workloads memory-bound. Kernel fusion is a key technique to a…
Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
Hui Zeng, Daming Zhao, Pengfei Yang +5
Generative reasoning with large language models (LLMs) often involves long decoding sequences, leading to substantial memory and latency overheads from accumulating key-value (KV)…
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
Hang Wu, Jianian Zhu, Yinghui Li +3
Large Language Models (LLMs) present a critical trade-off between inference quality and computational cost: larger models offer superior capabilities but incur significant latency,…
Jenga: Effective Memory Management for Serving LLM with Heterogeneity
Chen Zhang, Kuntai Du, Shu Liu +10
Large language models (LLMs) are widely used but expensive to run, especially as inference workloads grow. To lower costs, maximizing the request batch size by managing GPU memory…
FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
Jianian Zhu, Hang Wu, Haojie Wang +4
Multi-modal Large Language Models (MLLMs) serving systems commonly employ KV-cache compression to reduce memory footprint. However, existing compression methods introduce significa…
FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
Jiaao He, Jidong Zhai
Cost of serving large language models (LLM) is high, but the expensive and scarce GPUs are poorly efficient when generating tokens sequentially, unless the batch of sequences is en…