155 citations · 505 across the 15 of their papers we have counts for
15 papers
Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
Minyoung Huh, Brian Cheung, Pulkit Agrawal +1
This work examines the challenges of training neural networks using vector quantization using straight-through estimation. We find that a primary cause of training instability is t…
System identification of neural systems: If we got it right, would we know?
Yena Han, Tomaso Poggio, Brian Cheung
Artificial neural networks are being proposed as models of parts of the brain. The networks are compared to recordings of biological neurons, and good performance in reproducing ne…
Meta-Learning and Self-Supervised Pretraining for Real World Image Translation
Ileana Rugina, Rumen Dangovski, Mark Veillette +4
Recent advances in deep learning, in particular enabled by hardware advances and big data, have provided impressive results across a wide range of computational problems such as co…
Equivariant Contrastive Learning
Rumen Dangovski, Li Jing, Charlotte Loh +5
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations pres…
Compact and Optimal Deep Learning with Recurrent Parameter Generators
Jiayun Wang, Yubei Chen, Stella X. Yu +2
Deep learning has achieved tremendous success by training increasingly large models, which are then compressed for practical deployment. We propose a drastically different approach…
The Low-Rank Simplicity Bias in Deep Networks
Minyoung Huh, Hossein Mobahi, Richard Zhang +3
Modern deep neural networks are highly over-parameterized compared to the data on which they are trained, yet they often generalize remarkably well. A flurry of recent work has ask…