activity
20192025
most citedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

3.7k citations · 6.8k across the 18 of their papers we have counts for

collaborators
Showing 2021Show all

2 papers · 1 filter

cs.CL2021★ 30 cited

Counterfactual Memorization in Neural Language Models

Chiyuan Zhang, Daphne Ippolito, Katherine Lee +3

Modern neural language models that are widely used in various NLP tasks risk memorizing sensitive information from their training data. Understanding this memorization is important…

cs.CL2021★ 11 cited

Deduplicating Training Data Makes Language Models Better

Katherine Lee, Daphne Ippolito, Andrew Nystrom +4

We find that existing language modeling datasets contain many near-duplicate examples and long repetitive substrings. As a result, over 1% of the unprompted output of language mode…