16 citations · 32 across the 15 of their papers we have counts for
8 papers · 1 filter
Contrastive Entity Coreference and Disambiguation for Historical Texts
Abhishek Arora, Emily Silcock, Leander Heldring +1
Massive-scale historical document collections are crucial for social science research. Despite increasing digitization, these documents typically lack unique cross-document identif…
Newswire: A Large-Scale Structured Database of a Century of Historical News
Emily Silcock, Abhishek Arora, Luca D'Amico-Wong +1
In the U.S. historically, local newspapers drew their content largely from newswires like the Associated Press. Historians argue that newswires played a pivotal role in creating a…
News Deja Vu: Connecting Past and Present with Semantic Search
Brevin Franklin, Emily Silcock, Abhishek Arora +2
Social scientists and the general public often analyze contemporary events by drawing parallels with the past, a process complicated by the vast, noisy, and unstructured nature of…
LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models
Abhishek Arora, Melissa Dell
Linking information across sources is fundamental to a variety of analyses in social science, business, and government. While large language models (LLMs) offer enormous promise fo…
American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers
Melissa Dell, Jacob Carlson, Tom Bryan +7
Existing full text datasets of U.S. public domain newspapers do not recognize the often complex layouts of newspaper scans, and as a result the digitized content scrambles texts fr…
A Massive Scale Semantic Similarity Dataset of Historical English
Emily Silcock, Melissa Dell
A diversity of tasks use language models trained on semantic similarity data. While there are a variety of datasets that capture semantic similarity, they are either constructed fr…
Quantifying Character Similarity with Vision Transformers
Xinmei Yang, Abhishek Arora, Shao-Yu Jheng +1
Record linkage is a bedrock of quantitative social science, as analyses often require linking data from multiple, noisy sources. Off-the-shelf string matching methods are widely us…
Noise-Robust De-Duplication at Scale
Emily Silcock, Luca D'Amico-Wong, Jinglin Yang +1
Identifying near duplicates within large, noisy text corpora has a myriad of applications that range from de-duplicating training datasets, reducing privacy risk, and evaluating te…