1 citations · 4 across the 13 of their papers we have counts for
8 papers · 1 filter
Uncertainty Herding: One Active Learning Method for All Label Budgets
Wonho Bae, Gabriel L. Oliveira, Danica J. Sutherland
Most active learning research has focused on methods which perform well when many labels are available, but can be dramatically worse than random selection when label budgets are s…
Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics
Yi Ren, Danica J. Sutherland
Obtaining compositional mappings is important for the model to generalize well compositionally. To better understand when and how to encourage the model to learn such mappings, we…
Learning Representations for Independence Testing
Nathaniel Xu, Feng Liu, Danica J. Sutherland
Many tools exist to detect dependence between random variables, a core question across a wide range of machine learning, statistical, and scientific endeavors. Although several sta…
Generalized Coverage for More Robust Low-Budget Active Learning
Wonho Bae, Junhyug Noh, Danica J. Sutherland
The ProbCover method of Yehuda et al. is a well-motivated algorithm for active learning in low-budget regimes, which attempts to "cover" the data distribution with balls of a given…
Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition
Mohamad Amin Mohamadi, Zhiyuan Li, Lei Wu +1
We present a theoretical explanation of the ``grokking'' phenomenon, where a model generalizes long after overfitting,for the originally-studied problem of modular addition. First,…
Learning Dynamics of LLM Finetuning
Yi Ren, Danica J. Sutherland
Learning dynamics, which describes how the learning of specific training examples influences the model's predictions on other examples, gives us a powerful tool for understanding t…
Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
Yi Ren, Shangmin Guo, Linlu Qiu +2
With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase. Notably, recent advancements i…
Practical Kernel Tests of Conditional Independence
Roman Pogodin, Antonin Schrab, Yazhe Li +2
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence. A major challenge of conditional independence testing is to obtain the corre…