1 paper
Qimin Zhou, Haibo Liu, Yujie Wang +2
Foundation models are at the forefront of artificial intelligence. A tokenizer, converting the raw input into discrete representations that the model can understand, plays an impor…