108 citations · 415 across the 23 of their papers we have counts for
10 papers · 1 filter
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli +2
Understanding the dynamics of neural network parameters during training is one of the key challenges in building a theoretical foundation for deep learning. A central obstacle is t…
Conditional Negative Sampling for Contrastive Learning of Visual Representations
Mike Wu, Milan Mosse, Chengxu Zhuang +2
Recent methods for learning unsupervised visual representations, dubbed contrastive learning, optimize the noise-contrastive estimation (NCE) bound on mutual information between tw…
Identifying Learning Rules From Neural Network Observables
Aran Nayebi, Sanjana Srivastava, Surya Ganguli +1
The brain modifies its synaptic strengths during learning in order to better adapt to its environment. However, the underlying plasticity rules that govern learning are unknown. Ma…
Active World Model Learning with Progress Curiosity
Kuno Kim, Megumi Sano, Julian De Freitas +2
World models are self-supervised predictive models of how the world evolves. Humans learn world models by curiously exploring their environment, in the process acquiring compact ab…
Learning Physical Graph Representations from Visual Scenes
Daniel M. Bear, Chaofei Fan, Damian Mrowca +8
Convolutional Neural Networks (CNNs) have proved exceptional at learning representations for visual object categorization. However, CNNs do not explicitly encode objects, parts, an…
On Mutual Information in Contrastive Learning for Visual Representations
Mike Wu, Chengxu Zhuang, Milan Mosse +2
In recent years, several unsupervised, "contrastive" learning algorithms in vision have been shown to learn representations that perform remarkably well on transfer tasks. We show…
Pruning neural networks without any data by iteratively conserving synaptic flow
Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins +1
Pruning the parameters of deep neural networks has generated intense interest due to potential savings in time, memory and energy both during training and at test time. Recent work…
Visual Grounding of Learned Physical Models
Yunzhu Li, Toru Lin, Kexin Yi +5
Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. The abilities to perform physical…
Flexible and Efficient Long-Range Planning Through Curious Exploration
Aidan Curtis, Minjian Xin, Dilip Arumugam +2
Identifying algorithms that flexibly and efficiently discover temporally-extended multi-phase plans is an essential step for the advancement of robotics and model-based reinforceme…
Two Routes to Scalable Credit Assignment without Weight Symmetry
Daniel Kunin, Aran Nayebi, Javier Sagastuy-Brena +3
The neural plausibility of backpropagation has long been disputed, primarily for its use of non-local weight transport the biologically dubious requirement that one neuron inst…