25 citations · 50 across the 21 of their papers we have counts for
12 papers · 1 filter
SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
Tongcheng Fang, Hanling Zhang, Ruiqi Xie +8
Diffusion Transformers have demonstrated remarkable performance in video generation. However, their long input sequences incur substantial latency due to the quadratic complexity o…
db-SP: Accelerating Sparse Attention for Visual Generative Models with Dual-Balanced Sequence Parallelism
Siqi Chen, Ke Hong, Tianchen Zhao +4
Scaling Diffusion Transformer (DiT) inference via sequence parallelism is critical for reducing latency in visual generation, but is severely hampered by workload imbalance when ap…
LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation
Lianwei Yang, Haokun Lin, Tianchen Zhao +6
Diffusion Transformers (DiTs) have achieved impressive performance in text-to-image and text-to-video generation. However, their high computational cost and large parameter sizes p…
PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models
Tianchen Zhao, Ke Hong, Xinhao Yang +8
In visual generation, the quadratic complexity of attention mechanisms results in high memory and computational costs, especially for longer token sequences required in high-resolu…
LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization
Rui Xie, Tianchen Zhao, Zhihang Yuan +5
Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models. However, c…
DiTFastAttn: Attention Compression for Diffusion Transformer Models
Zhihang Yuan, Hanling Zhang, Pu Lu +6
Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to the quadratic complexity of self-attention operators. We propose DiTFastAt…
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
Tianchen Zhao, Tongcheng Fang, Haofeng Huang +10
Diffusion transformers have demonstrated remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, la…
MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization
Tianchen Zhao, Xuefei Ning, Tongcheng Fang +6
Diffusion models have achieved significant visual generation quality. However, their significant computational and memory costs pose challenge for their application on resource-con…
FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models
Lin Zhao, Tianchen Zhao, Zinan Lin +4
In recent years, there has been significant progress in the development of text-to-image generative models. Evaluating the quality of the generative models is one essential step in…
Ada3D : Exploiting the Spatial Redundancy with Adaptive Inference for Efficient 3D Object Detection
Tianchen Zhao, Xuefei Ning, Ke Hong +8
Voxel-based methods have achieved state-of-the-art performance for 3D object detection in autonomous driving. However, their significant computational and memory costs pose a chall…
CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric Guidance
Tianchen Zhao, Niansong Zhang, Xuefei Ning +3
Transformers have gained much attention by outperforming convolutional neural networks in many 2D vision tasks. However, they are known to have generalization problems and rely on…
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation
Xuefei Ning, Tianchen Zhao, Wenshuo Li +3
Budgeted pruning is the problem of pruning under resource constraints. In budgeted pruning, how to distribute the resources across layers (i.e., sparsity allocation) is the key pro…