Autoregressive Large Language Models are Computationally Universal

Dale Schuurmans, Hanjun Dai, Francesco Zanini

We show that autoregressive decoding of a transformer-based language model can realize universal computation, without external intervention or modification of the model's weights.…