Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners

Shashank Gupta, Subhabrata Mukherjee, Krishan Subudhi +4

Traditional multi-task learning (MTL) methods use dense networks that use the same set of shared weights across several different tasks. This often creates interference where two o…