Tokenization Constraints in LLMs: A Study of Symbolic and Arithmetic Reasoning Limits

Xiang Zhang, Juntai Cao, Jiaqi Wei +2

Tokenization is the first - and often underappreciated - layer of computation in language models. While Chain-of-Thought (CoT) prompting enables transformer models to approximate r…