Comparing Variation in Tokenizer Outputs Using a Series of Problematic and Challenging Biomedical Sentences

Christopher Meaney, Therese A Stukel, Peter C Austin +1

Background & Objective: Biomedical text data are increasingly available for research. Tokenization is an initial step in many biomedical text mining pipelines. Tokenization is the…