Transformer Vs. MLP-Mixer: Exponential Expressive Gap For NLP Problems

Dan Navon, Alex M. Bronstein

Vision-Transformers are widely used in various vision tasks. Meanwhile, there is another line of works starting with the MLP-mixer trying to achieve similar performance using mlp-b…