Change of Thought: Adaptive Test-Time Computation

Mrinal Mathur, Mike Doan, Barak Pearlmutter +1

Transformers evaluated in a single, fixed-depth pass are provably limited in expressive power to the constant-depth circuit class TC0. Running a Transformer autoregressively remove…