2 citations · 2 across the 5 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
cs.LG2026
MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
Jianlin Yu, Jing Lin, Linghui Kong +13
The quadratic cost of attention is a major bottleneck in diffusion-based video generation models. MXFP4 attention provides a promising path toward efficient inference, but direct M…
cs.LG2026
StreamKL: Fast and Memory-Efficient KL Divergence for Boosting Attention Distillation
Guangda Liu, Yiquan Wang, Chengwei Li +6
Attention distillation, which trains one attention distribution to match another by minimizing their Kullback-Leibler (KL) divergence, is widely used in knowledge distillation, mod…
cs.LG2026
Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
Longteng Zhang, Sen Wu, Shuai Hou +7
Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in…