Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization

Dixuan Wang, Yanda Li, Junyuan Jiang +5

Large Language Models (LLMs) have shown remarkable capabilities in language understanding and generation. Nonetheless, it was also witnessed that LLMs tend to produce inaccurate re…