Shortcut Solutions Learned by Transformers Impair Continual Compositional Reasoning

William T. Redman, Erik C. Johnson, Brian Robinson

Identifying and exploiting common features across domains is at the heart of the human ability to make analogies, and is believed to be crucial for the ability to continually learn…