Understanding MLP-Mixer as a Wide and Sparse MLP

Tomohiro Hayase, Ryo Karakida

Multi-layer perceptron (MLP) is a fundamental component of deep learning, and recent MLP-based architectures, especially the MLP-Mixer, have achieved significant empirical success.…