Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts

Strahinja Nikolic, Ilker Oguz, Demetri Psaltis

Understanding the internal organization of neural networks remains a fundamental challenge in deep learning interpretability. We address this challenge by exploring a novel Sparse…