1 citations · 4 across the 13 of their papers we have counts for
15 papers
Memorize, Adapt, Ignore: Diagnosing Robot Learning Mechanisms under Training Data Variation
Ke Zhang, Danica J. Sutherland, Chao Liu
Training data variation, whether through designing a domain randomization (DR) scheme in simulation or curating demonstrations for imitation learning, is a primary lever for improv…
Understanding Generalization Requires Universal Induction
Aram Ebtekar, Marcus Hutter, Danica J. Sutherland
Classical statistical theory is insufficient to explain the successes of general-purpose AI models, because it depends on handcrafted inductive biases that it cannot justify. No Fr…
Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation
Jeongin Kim, Wonho Bae, YouLee Han +4
Semantic segmentation demands dense pixel-level annotations, which can be prohibitively expensive - especially under extremely constrained labeling budgets. In this paper, we addre…
DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing
Zhijian Zhou, Xunye Tian, Liuhua Peng +4
To adapt kernel two-sample and independence testing to complex structured data, aggregation of multiple kernels is frequently employed to boost testing power compared to single-ker…
Uncertainty Herding: One Active Learning Method for All Label Budgets
Wonho Bae, Gabriel L. Oliveira, Danica J. Sutherland
Most active learning research has focused on methods which perform well when many labels are available, but can be dramatically worse than random selection when label budgets are s…
Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics
Yi Ren, Danica J. Sutherland
Obtaining compositional mappings is important for the model to generalize well compositionally. To better understand when and how to encourage the model to learn such mappings, we…
Learning Representations for Independence Testing
Nathaniel Xu, Feng Liu, Danica J. Sutherland
Many tools exist to detect dependence between random variables, a core question across a wide range of machine learning, statistical, and scientific endeavors. Although several sta…
Generalized Coverage for More Robust Low-Budget Active Learning
Wonho Bae, Junhyug Noh, Danica J. Sutherland
The ProbCover method of Yehuda et al. is a well-motivated algorithm for active learning in low-budget regimes, which attempts to "cover" the data distribution with balls of a given…
Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition
Mohamad Amin Mohamadi, Zhiyuan Li, Lei Wu +1
We present a theoretical explanation of the ``grokking'' phenomenon, where a model generalizes long after overfitting,for the originally-studied problem of modular addition. First,…
Learning Dynamics of LLM Finetuning
Yi Ren, Danica J. Sutherland
Learning dynamics, which describes how the learning of specific training examples influences the model's predictions on other examples, gives us a powerful tool for understanding t…
Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
Yi Ren, Shangmin Guo, Linlu Qiu +2
With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase. Notably, recent advancements i…
Practical Kernel Tests of Conditional Independence
Roman Pogodin, Antonin Schrab, Yazhe Li +2
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence. A major challenge of conditional independence testing is to obtain the corre…
AdaFlood: Adaptive Flood Regularization
Wonho Bae, Yi Ren, Mohamad Osama Ahmed +3
Although neural networks are conventionally optimized towards zero training loss, it has been recently learned that targeting a non-zero training loss threshold, referred to as a f…
Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling
Wonho Bae, Jing Wang, Danica J. Sutherland
Most meta-learning methods assume that the (very small) context set used to establish a new task at test time is passively provided. In some settings, however, it is feasible to ac…
Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddings
Yi Ren, Samuel Lavoie, Mikhail Galkin +2
Compositional generalization, the ability of an agent to generalize to unseen combinations of latent factors, is easy for humans but hard for deep neural networks. A line of resear…