4 papers
ParamSpMM: Adaptive and Efficient Sparse Matrix-Matrix Multiplication on GPUs for GNNs
Lixing Zhang, Guanhua Ye, Hongzheng Li +2
Fueled by the ability to mine real-world graph data, GNN applications have experienced phenomenal growth. Sparse Matrix-Matrix Multiplication (SpMM) is a critical operator in GNNs.…
Libra: Unleashing GPU Heterogeneity for High-Performance Sparse Matrix Multiplication
Jinliang Shi, Shigang Li, Youxuan Xu +4
Sparse matrix multiplication operators (i.e., SpMM and SDDMM) are widely used in deep learning and scientific computing. Modern accelerators are commonly equipped with Tensor Core…
SparkAttention: High-Performance Multi-Head Attention for Large Models on Volta GPU Architecture
Youxuan Xu, Tong Wu, Shigang Li +2
Transformer are widely used in various fields such as natural language processing and computer vision. However, the training time for large Transformer models can be challenging du…
FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores
Jinliang Shi, Shigang Li, Youxuan Xu +3
Sparse Matrix-matrix Multiplication (SpMM) and Sampled Dense-dense Matrix Multiplication (SDDMM) are important sparse operators in scientific computing and deep learning. Tensor Co…