3 papers
cs.CL2026
Comparing Transformers and Hybrid Models at the Token Level
Yanhong Li, William Merrill
Hybrid language models that mix attention and recurrent layers have shown promise: theoretically, recurrent layers ameliorate the limitations of pure transformers on state tracking…
cs.LG2026
Efficiently Representing Algorithms With Chain-of-Thought Transformers
Yanhong Li, Anej Svete, Ashish Sabharwal +1
The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by…
cs.AI2026
Training Transformers as a Universal Computer
Ruize Xu, Chenxiao Yang, Yanhong Li +1
We demonstrate that a small transformer can learn to execute programs in MicroPy, a simplified yet computationally universal programming language. Given procedure definitions toget…