Extrapolation by Association: Length Generalization Transfer in Transformers

Ziyang Cai, Nayoung Lee, Avi Schwarzschild +2

Transformer language models have demonstrated impressive generalization capabilities in natural language domains, yet we lack a fine-grained understanding of how such generalizatio…