3 papers
cs.LG2026
Learning to Detect Language Model Training Data via Active Reconstruction
Junjie Oscar Yin, John X. Morris, Vitaly Shmatikov +2
Detecting LLM training data is generally framed as a membership inference attack (MIA) problem. However, conventional MIAs operate passively on fixed model weights, using log-likel…
cs.CL2025
Approximating Language Model Training Data from Weights
John X. Morris, Junjie Oscar Yin, Woojeong Kim +2
Modern language models often have open weights but closed training data. We formalize the problem of data approximation from model weights and propose several baselines and metrics…
cs.LG2024
Compute-Constrained Data Selection
Junjie Oscar Yin, Alexander M. Rush
Data selection can reduce the amount of training data needed to finetune LLMs; however, the efficacy of data selection scales directly with its compute. Motivated by the practical…