Speculating LLMs' Chinese Training Data Pollution from Their Tokens

Qingjie Zhang, Di Wang, Haoting Qian +7

Tokens are basic elements in the datasets for LLM training. It is well-known that many tokens representing Chinese phrases in the vocabulary of GPT (4o/4o-mini/o1/o3/4.5/4.1/o4-min…