The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models

Adrian Cosma, Stefan Ruseti, Emilian Radoi +1

Despite their remarkable progress across diverse domains, Large Language Models (LLMs) consistently fail at simple character-level tasks, such as counting letters in words, due to…