Accelerating Transformer Inference and Training with 2:4 Activation Sparsity

Daniel Haziza, Timothy Chou, Dhruv Choudhary +7

In this paper, we demonstrate how to leverage 2:4 sparsity, a popular hardware-accelerated GPU sparsity pattern, to activations to accelerate large language model training and infe…