PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3

Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation. This limi…