3 citations · 3 across the 10 of their papers we have counts for
10 papers
SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion
Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei +3
Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full…
Correcting test set contamination by spiking the training data
Johnny Tian-Zheng Wei, Jerry Li, Ameya Godbole +1
The literature on test set contamination largely focuses on detection, but the correction of contaminated test scores is underexplored. Our core proposal is to spike the training d…
Psychological Steering of Large Language Models
Leonardo Blas, Robin Jia, Emilio Ferrara
Large language models (LLMs) emulate a consistent human-like behavior that can be shaped through activation-level interventions. This paradigm is converging on additive residual-st…
Hubble: a Model Suite to Advance the Study of LLM Memorization
Johnny Tian-Zheng Wei, Ameya Godbole, Mohammad Aflah Khan +7
We present Hubble, a suite of fully open-source large language models (LLMs) for the scientific study of LLM memorization. Hubble models come in standard and perturbed variants: st…
TokenSmith: Streamlining Data Editing, Search, and Inspection for Large-Scale Language Model Training and Interpretability
Mohammad Aflah Khan, Ameya Godbole, Johnny Tian-Zheng Wei +5
Understanding the relationship between training data and model behavior during pretraining is crucial, but existing workflows make this process cumbersome, fragmented, and often in…
Teaching Models to Understand (but not Generate) High-risk Data
Ryan Wang, Matthew Finlayson, Luca Soldaini +2
Language model developers typically filter out high-risk content -- such as toxic or copyrighted text -- from their pre-training data to prevent models from generating similar outp…
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge
Xinyue Cui, Johnny Tian-Zheng Wei, Swabha Swayamdipta +1
Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track a…
Interrogating LLM design under a fair learning doctrine
Johnny Tian-Zheng Wei, Maggie Wang, Ameya Godbole +2
The current discourse on large language models (LLMs) and copyright largely takes a "behavioral" perspective, focusing on model outputs and evaluating whether they are substantiall…
Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics
Ameya Godbole, Robin Jia
Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge…
Proving membership in LLM pretraining data via data watermarks
Johnny Tian-Zheng Wei, Ryan Yixiang Wang, Robin Jia
Detecting whether copyright holders' works were used in LLM pretraining is poised to be an important problem. This work proposes using data watermarks to enable principled detectio…