Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers

Rya Sanovar, Srikant Bharadwaj, Renee St. Amant +2

Transformer-based models have emerged as one of the most widely used architectures for natural language processing, natural language generation, and image generation. The size of t…