ReduceFormer: Attention with Tensor Reduction by Summation

John Yang, Le An, Su Inn Park

Transformers have excelled in many tasks including vision. However, efficient deployment of transformer models in low-latency or high-throughput applications is hindered by the com…