Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement

Catherine Arnett, Pamela D. Rivière, Tyler A. Chang +1

The relationship between language model tokenization and performance is an open area of research. Here, we investigate how different tokenization schemes impact number agreement in…