Internal Knowledge Without External Expression: Probing the Generalization Boundary of a Classical Chinese Language Model

Jiuting Chen, Yuan Lian, Hao Wu +4

We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.56 billion tokens of pure Classical Chinese, with zero English characters or Arabic numer…