Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts

Erik Daxberger, Floris Weers, Bowen Zhang +7

Sparse Mixture-of-Experts models (MoEs) have recently gained popularity due to their ability to decouple model size from inference efficiency by only activating a small subset of t…