Transformers are Universal In-context Learners

Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré

Transformers are deep architectures that define "in-context mappings" which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a s…