2 citations · 2 across the 7 of their papers we have counts for
7 papers
A mathematical theory of balancing relational generalization and memorization
Luke Cheng, Samuel Lippl
Humans, animals, and modern machine learning models exhibit impressive abilities to learn complex behaviors and generalize these behaviors to unseen situations. This ability requir…
A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4
Pretraining and fine-tuning are central stages in modern machine learning systems. In practice, feature learning plays an important role across both stages: deep neural networks le…
AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models
Samuel Lippl, Thomas McGee, Kimberly Lopez +5
How do inference time and latent computations enable large language models (LLMs) to solve multi-step reasoning problems? We introduce AlgoTrace, a framework for tracing and steeri…
When does compositional structure yield compositional generalization? A kernel theory
Samuel Lippl, Kim Stachenfeld
Compositional generalization (the ability to respond correctly to novel combinations of familiar components) is thought to be a cornerstone of intelligent behavior. Compositionally…
Source Invariance and Probabilistic Transfer: A Testable Theory of Probabilistic Neural Representations
Samuel Lippl, Raphael Gerraty, John Morrison +1
As animals interact with their environments, they must infer properties of their surroundings. Some animals, including humans, can represent uncertainty about those properties. But…
Inductive biases of multi-task learning and finetuning: multiple regimes of feature reuse
Samuel Lippl, Jack W. Lindsey
Neural networks are often trained on multiple tasks, either simultaneously (multi-task learning, MTL) or sequentially (pretraining and subsequent finetuning, PT+FT). In particular,…
The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks
Samuel Lippl, L. F. Abbott, SueYeon Chung
Understanding the asymptotic behavior of gradient-descent training of deep neural networks is essential for revealing inductive biases and improving network performance. We derive…