Winner-Take-All bottlenecks enforce disentangled symbolic representations in multi-task learning

Julian Gutheil, Simon Hitzginger, Robert Legenstein

Winner-take-all (WTA) networks constitute a central circuit motif in cortical networks of the brain. In addition, WTA-like activations are abundant in modern deep learning models i…