Automatic Channel Pruning for Multi-Head Attention

Eunho Lee, Youngbae Hwang

Despite the strong performance of Transformers, their quadratic computation complexity presents challenges in applying them to vision tasks. Automatic pruning is one of effective m…