6 papers
DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding
Junqing Lin, Jingwei Sun, Guangzhong Sun
Linear attention models eliminate the quadratic prefix computation and context-growing KV cache of softmax attention by replacing pairwise token interactions with recurrent state u…
FlashQuant: Sparse-Dense Fusion for Memory-Efficient Outlier-Aware LLM Inference
Junqing Lin, Jingwei Sun, Zhengding Hu +1
Low-bit quantization reduces the memory footprint and computational cost of large language model (LLM) inference. However, high-magnitude outlier weights can induce substantial qua…
Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling
Xingyu Zheng, Xianglong Liu, Yifu Ding +4
Hardware-agnostic strategies for accelerating text-to-image diffusion, such as timestep distillation and feature caching, can reduce inference time without custom kernels or system…
Beyond a Single Queue: Multi-Level-Multi-Queue as an Effective Design for SSSP problems on GPUs
Zhengding Hu, Jingwen Sun, Le Jiang +4
As one of the most fundamental problems in graph processing, the Single-Source Shortest Path (SSSP) problem plays a critical role in numerous application scenarios. However, existi…
Token Pruning for In-Context Generation in Diffusion Transformers
Junqing Lin, Xingyu Zheng, Pei Cheng +3
In-context generation significantly enhances Diffusion Transformers (DiTs) by enabling controllable image-to-image generation through reference examples. However, the resulting inp…
Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storage
Junqing Lin, Jingwei Sun, Mingge Lu +1
Sparse Matrix-Vector Multiplication (SpMV) has become a critical performance bottleneck in the local deployment of sparse Large Language Models (LLMs), where inference predominantl…