Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks

Yuxuan Li, James L. McClelland

Transformer networks have seen great success in natural language processing and machine vision, where task objectives such as next word prediction and image classification benefit…