On consistency scores in text data with an implementation in R

Ke-Li Chiu, Rohan Alexander

In this paper, we introduce a reproducible cleaning process for the text extracted from PDFs using n-gram models. Our approach compares the originally extracted text with the text…