More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing

Sagi Shaier, Francisco Pereira, Katharina von der Wense +2

The evolution of biological neural systems has led to both modularity and sparse coding, which enables energy efficiency and robustness across the diversity of tasks in the lifespa…