From the 1 of 16 linked papers with an AI index.
4 papers · 1 filter
RT-Lynx: Putting GEMM Sparsity in the Right Place for Diffusion Models
Xing Cong, Hanlin Tang, Kan Liu +4
Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs. While prior work has reduced this cost via quantization and disti…
Shiva-DiT: Residual-Based Differentiable Top- Selection for Efficient Diffusion Transformers
Jiaji Zhang, Hailiang Zhao, Guoxuan Zhu +9
Diffusion Transformers (DiTs) incur prohibitive computational costs due to the quadratic scaling of self-attention. Existing pruning methods fail to simultaneously satisfy differen…
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
Xing Li, Zeyu Xing, Yiming Li +6
KV cache quantization can improve Large Language Models (LLMs) inference throughput and latency in long contexts and large batch-size scenarios while preserving LLMs effectiveness.…
Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony
Han Lu, Zichen Liu, Shaopan Xiong +19
Synchronous Reinforcement Learning (RL) post-training has emerged as a crucial step for enhancing Large Language Models (LLMs) with diverse capabilities. However, many systems desi…