4 papers · 1 filter
Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functions
Steve Hong, Runa Eschenhagen, Bruno Mlodozeniec +1
Influence functions offer a principled way to trace model predictions back to training data, but their use in deep learning is hampered by the need to invert a large, ill-condition…
Distributional Training Data Attribution: What do Influence Functions Sample?
Bruno Mlodozeniec, Isaac Reid, Sam Power +4
Randomness is an unavoidable part of training deep learning models, yet something that traditional training data attribution algorithms fail to rigorously account for. They ignore…
Fresh in memory: Training-order recency is linearly encoded in language model activations
Dmitrii Krasheninnikov, Richard E. Turner, David Krueger
We show that language models' activations linearly encode when information was learned during training. Our setup involves creating a model with a known training order by sequentia…
Influence Functions for Scalable Data Attribution in Diffusion Models
Bruno Mlodozeniec, Runa Eschenhagen, Juhan Bae +3
Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In th…