37 citations · 122 across the 12 of their papers we have counts for
17 papers · 1 filter
SageBwd: A Trainable Low-bit Attention
Jintao Zhang, Marco Chen, Haoxu Wang +5
Low-bit attention, such as SageAttention, has emerged as an effective approach for accelerating model inference, but its applicability to training remains poorly understood. In pri…
CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
Weiyu Huang, Yuezhou Hu, Jun Zhu +1
Sparsity-aware training is an effective approach for transforming large language models (LLMs) into hardware-friendly sparse patterns, thereby reducing latency and memory consumpti…
Efficient Hyperparameter Tuning via Trajectory Invariance Principle
Bingrui Li, Jiaxin Wen, Zhanpeng Zhou +2
As hyperparameter tuning becomes increasingly costly at scale, efficient tuning methods are essential. Yet principles for guiding hyperparameter tuning remain limited. In this work…
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
Jintao Zhang, Haoxu Wang, Kai Jiang +10
In Diffusion Transformer (DiT) models, particularly for video generation, attention latency is a major bottleneck due to the long sequence length and the quadratic complexity. We f…
Accurate INT8 Training Through Dynamic Block-Level Fallback
Pengle Zhang, Jia Wei, Jintao Zhang +2
Transformer models have achieved remarkable success across various AI applications but face significant training costs. Low-bit training, such as INT8 training, can leverage comput…
Identifying Sensitive Weights via Post-quantization Integral
Yuezhou Hu, Weiyu Huang, Zichen Liang +4
Serving Large Language Models (LLMs) is costly. However, post-training weight quantization can address this problem by both compressing their sizes for limited memory and saving ba…