AttMEMO : Accelerating Transformers with Memoization on Big Memory Systems

Yuan Feng, Hyeran Jeon, Filip Blagojevic +3

Transformer models gain popularity because of their superior inference accuracy and inference throughput. However, the transformer is computation-intensive, causing a long inferenc…