14 papers · 1 filter
Adversarial Gradient Forcing for Robust Few-Step Autoregressive Video Generation
Chenxi Xia, Yixuan Xu, Shan Liu +7
Autoregressive video generation models generate videos sequentially, with each new block conditioned on previously generated blocks. By training on the model's own rollouts, Self F…
Adaptive Context Parallelism for Production LLM Serving
Jiarui Guo, Rongle Wang, Peijun Huang +7
As LLM context windows expand and input sequences grow longer, serving systems face increasing computational and memory demands. Context parallelism (CP), which partitions the inpu…
SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
Qi Zhao, Qirui Li, Hanlin Tang +10
Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from bloc…
Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
Maohua Li, Qirui Li, Yanke Zhou +10
Modern text-to-image diffusion transformers (DiTs) generate images through joint attention, in which text and image tokens interact directly within a single sequence. In large-scal…
CKV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Chuheng Du, Junyi Chen, Hanlin Tang +7
Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning. To mitigate the growing inf…
Rethinking Cross-Layer Information Routing in Diffusion Transformers
Chao Xu, Maohua Li, Qirui Li +9
Diffusion Transformers (DiTs) have become a de facto backbone of modern visual generation, and nearly every major axis of their design -- tokenization, attention, conditioning, obj…
Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps
Yanke Zhou, Yiduo Li, Hanlin Tang +6
Long-context inference in large language models is bottlenecked by the quadratic cost of full attention. Existing efficient alternatives often rely either on native sparse training…
RTP-LLM: High-Performance Alibaba LLM Inference Engine
Boyu Tan, Jiarui Guo, Zongwei Lv +26
Large Language Models (LLMs) have revolutionized AI applications, but deploying them at scale presents significant challenges. We present RTP-LLM, a high-performance inference engi…
Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models
Kesong Li, Yixuan Xu, Kuo-kun Tseng +3
Direct Preference Optimization (DPO) is successful for alignment in LLMs but still faces challenges in text-to-image generation. Existing studies are confined to denoising diffusio…
RT-Lynx: Putting GEMM Sparsity in the Right Place for Diffusion Models
Xing Cong, Hanlin Tang, Kan Liu +3
Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs. While prior work has reduced this cost via quantization and disti…
LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows
Lingyun Yang, Suyi Li, Tianyu Feng +10
Text-to-image generation executes a diffusion workflow comprising multiple models centered on a base diffusion model. Existing serving systems treat each workflow as an opaque mono…
Rethinking Retrieval-Augmentation as Synthesis: A Query-Aware Context Merging Approach
Jiarui Guo, Yuemeng Xu, Zongwei Lv +6
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to extend their existing knowledge by dynamically incorporating external information. However, practical d…
Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
Xianzhe Zheng, Zhengheng Wang, Ruiyan Ma +17
The memory-for-computation paradigm of KV caching is essential for accelerating large language model (LLM) inference service, but limited GPU high-bandwidth memory (HBM) capacity m…
Shiva-DiT: Residual-Based Differentiable Top- Selection for Efficient Diffusion Transformers
Jiaji Zhang, Hailiang Zhao, Jiaju Wu +3
Diffusion Transformers (DiTs) are costly at high resolution because self-attention scales quadratically with token sequence length. Existing pruning methods do not jointly provide…