90 citations · 332 across the 63 of their papers we have counts for
8 papers · 1 filter
Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective
Tanya Marwah, Zachary C. Lipton, Jianfeng Lu +1
A burgeoning line of research leverages deep neural networks to approximate the solutions to high dimensional PDEs, opening lines of theoretical inquiry focused on explaining how i…
Statistical Efficiency of Score Matching: The View from Isoperimetry
Frederic Koehler, Alexander Heckett, Andrej Risteski
Deep generative models parametrized up to a normalizing constant (e.g. energy-based models) are difficult to train by maximizing the likelihood of the data because the likelihood a…
Pitfalls of Gaussians as a noise distribution in NCE
Holden Lee, Chirag Pabbaraju, Anish Sevekari +1
Noise Contrastive Estimation (NCE) is a popular approach for learning probability density functions parameterized up to a constant of proportionality. The main idea is to design a…
Contrasting the landscape of contrastive and non-contrastive learning
Ashwini Pokle, Jinjin Tian, Yuchen Li +1
A lot of recent advances in unsupervised feature learning are based on designing features which are invariant under semantic data augmentations. A common way to do this is contrast…
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions
Binghui Peng, Andrej Risteski
Continual learning is an emerging paradigm in machine learning, wherein a model is exposed in an online fashion to data from multiple different distributions (i.e. environments), a…
Masked prediction tasks: a parameter identifiability view
Bingbin Liu, Daniel Hsu, Pradeep Ravikumar +1
The vast majority of work in self-supervised learning, both theoretical and empirical (though mostly the latter), have largely focused on recovering good features for downstream ta…
Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
Frederic Koehler, Holden Lee, Andrej Risteski
We consider Ising models on the hypercube with a general interaction matrix , and give a polynomial time sampling algorithm when all but eigenvalues of lie in an inte…
Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization
Elan Rosenfeld, Pradeep Ravikumar, Andrej Risteski
A common explanation for the failure of deep networks to generalize out-of-distribution is that they fail to recover the "correct" features. We challenge this notion with a simple…