Diversity-Guided MLP Reduction for Efficient Large Vision Transformers

Chengchao Shen, Hourun Zhu, Gongfan Fang +2

Transformer models achieve excellent scaling property, where the performance is improved with the increment of model capacity. However, large-scale model parameters lead to an unaf…