16 citations · 32 across the 15 of their papers we have counts for
15 papers
A Unifying Framework for Robust and Efficient Inference with Unstructured Data
Jacob Carlson, Melissa Dell
To analyze unstructured data (text, images, audio, video), economists typically first extract low-dimensional structured features with a neural network. Neural networks do not make…
Contrastive Entity Coreference and Disambiguation for Historical Texts
Abhishek Arora, Emily Silcock, Leander Heldring +1
Massive-scale historical document collections are crucial for social science research. Despite increasing digitization, these documents typically lack unique cross-document identif…
Newswire: A Large-Scale Structured Database of a Century of Historical News
Emily Silcock, Abhishek Arora, Luca D'Amico-Wong +1
In the U.S. historically, local newspapers drew their content largely from newswires like the Associated Press. Historians argue that newswires played a pivotal role in creating a…
News Deja Vu: Connecting Past and Present with Semantic Search
Brevin Franklin, Emily Silcock, Abhishek Arora +2
Social scientists and the general public often analyze contemporary events by drawing parallels with the past, a process complicated by the vast, noisy, and unstructured nature of…
EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge
Tom Bryan, Jacob Carlson, Abhishek Arora +1
Billions of public domain documents remain trapped in hard copy or lack an accurate digitization. Modern natural language processing methods cannot be used to index, retrieve, and…
LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models
Abhishek Arora, Melissa Dell
Linking information across sources is fundamental to a variety of analyses in social science, business, and government. While large language models (LLMs) offer enormous promise fo…
American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers
Melissa Dell, Jacob Carlson, Tom Bryan +7
Existing full text datasets of U.S. public domain newspapers do not recognize the often complex layouts of newspaper scans, and as a result the digitized content scrambles texts fr…
A Massive Scale Semantic Similarity Dataset of Historical English
Emily Silcock, Melissa Dell
A diversity of tasks use language models trained on semantic similarity data. While there are a variety of datasets that capture semantic similarity, they are either constructed fr…
Quantifying Character Similarity with Vision Transformers
Xinmei Yang, Abhishek Arora, Shao-Yu Jheng +1
Record linkage is a bedrock of quantitative social science, as analyses often require linking data from multiple, noisy sources. Off-the-shelf string matching methods are widely us…
Linking Representations with Multimodal Contrastive Learning
Abhishek Arora, Xinmei Yang, Shao-Yu Jheng +1
Many applications require linking individuals, firms, or locations across datasets. Most widely used methods, especially in social science, do not employ deep learning, with record…
Efficient OCR for Building a Diverse Digital History
Jacob Carlson, Tom Bryan, Melissa Dell
Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence archite…
Noise-Robust De-Duplication at Scale
Emily Silcock, Luca D'Amico-Wong, Jinglin Yang +1
Identifying near duplicates within large, noisy text corpora has a myriad of applications that range from de-duplicating training datasets, reducing privacy risk, and evaluating te…
LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis
Zejiang Shen, Ruochen Zhang, Melissa Dell +3
Recent advances in document image analysis (DIA) have been primarily driven by the application of neural networks. Ideally, research outcomes could be easily deployed in production…
OLALA: Object-Level Active Learning for Efficient Document Layout Annotation
Zejiang Shen, Jian Zhao, Melissa Dell +2
Document images often have intricate layout structures, with numerous content regions (e.g. texts, figures, tables) densely arranged on each page. This makes the manual annotation…
A Large Dataset of Historical Japanese Documents with Complex Layouts
Zejiang Shen, Kaixuan Zhang, Melissa Dell
Deep learning-based approaches for automatic document layout analysis and content extraction have the potential to unlock rich information trapped in historical documents on a larg…