126 citations
- Huawei Technologies (China)CN21 papers
- Peking UniversityCN10 papers
- Huawei Technologies (Sweden)SE7 papers
- Huawei Technologies (United States)US6 papers
- University College LondonGB5 papers
- University of Hong KongHK5 papers
- Hong Kong University of Science and TechnologyHK3 papers
- Lomonosov Moscow State UniversityRU3 papers
- National Research University Higher School of EconomicsRU3 papers
- Shanghai Jiao Tong UniversityCN3 papers
- Southern University of Science and TechnologyCN3 papers
- Tsinghua UniversityCN3 papers
Showing 2026 · cs.LGShow all
2 papers · 2 filters
cs.LG2026
EFQ-Softmax: Exp-Free Quantization for Softmax
Haohui Han, Yuming Wan, Hongni Wang +4
Low-bit attention accelerates Transformer inference by moving the and matrix multiplications to FP8 or FP4 matrix engines. However, the softmax path often evaluates…
cs.LG2026
Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
Longteng Zhang, Sen Wu, Shuai Hou +7
Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in…