932 citations · 2.2k across the 38 of their papers we have counts for
8 papers · 1 filter
Are all negatives created equal in contrastive instance discrimination?
Tiffany Tianhui Cai, Jonathan Frankle, David J. Schwab +1
Self-supervised learning has recently begun to rival supervised learning on computer vision tasks. Many of the recent approaches have been based on contrastive instance discriminat…
Learning Optimal Representations with the Decodable Information Bottleneck
Yann Dubois, Douwe Kiela, David J. Schwab +1
We address the question of characterizing and finding optimal representations for supervised learning. Traditionally, this question has been tackled using the Information Bottlenec…
Superlinear Precision and Memory in Simple Population Codes
Jimmy H. J. Kim, Ila Fiete, David J. Schwab
The brain constructs population codes to represent stimuli through widely distributed patterns of activity across neurons. An important figure of merit of population codes is how m…
What makes it possible to learn probability distributions in the natural world?
William Bialek, Stephanie E. Palmer, David J. Schwab
Organisms and algorithms learn probability distributions from previous observations, either over evolutionary time or on the fly. In the absence of regularities, estimating the und…
Theory of gating in recurrent neural networks
Kamesh Krishnamurthy, Tankut Can, David J. Schwab
Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Prior theoretical work has focused on RNNs with additive inte…
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
Jonathan Frankle, David J. Schwab, Ari S. Morcos
A wide variety of deep learning techniques from style transfer to multitask learning rely on training affine transformations of features. Most prominent among these is the popular…