Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)

Lena Strobl, Dana Angluin, Robert Frank

While transformers have proven enormously successful in a range of tasks, their fundamental properties as models of computation are not well understood. This paper contributes to t…