Sparse Competition during Training For the Emergence of Specialized Modules

Baptiste Rossigneux, Karim Haroun

Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representations and reducing redundancy. I…