Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

Oliver Kraus, Yash Sarrof, Yuekun Yao +2

Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness. However, whether Trans…