Beyond 2:4: exploring V:N:M sparsity for efficient transformer inference on GPUs

Kang Zhao, Tao Yuan, Han Bao +6

To date, 2:4 sparsity has stood as the only sparse pattern that can be accelerated using sparse tensor cores on GPUs. In practice, 2:4 sparsity often possesses low actual speedups…