Specialised or Generic? Tokenization Choices for Radiology Language Models

Hermione Warr, Wentian Xu, Harry Anthony +3

The vocabulary used by language models (LM) - defined by the tokenizer - plays a key role in text generation quality. However, its impact remains under-explored in radiology. In th…