16 citations · 35 across the 15 of their papers we have counts for
24 papers
Contrasting the landscape of contrastive and non-contrastive learning
Ashwini Pokle, Jinjin Tian, Yuchen Li +1
A lot of recent advances in unsupervised feature learning are based on designing features which are invariant under semantic data augmentations. A common way to do this is contrast…
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions
Binghui Peng, Andrej Risteski
Continual learning is an emerging paradigm in machine learning, wherein a model is exposed in an online fashion to data from multiple different distributions (i.e. environments), a…
Masked prediction tasks: a parameter identifiability view
Bingbin Liu, Daniel Hsu, Pradeep Ravikumar +1
The vast majority of work in self-supervised learning, both theoretical and empirical (though mostly the latter), have largely focused on recovering good features for downstream ta…
Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
Frederic Koehler, Holden Lee, Andrej Risteski
We consider Ising models on the hypercube with a general interaction matrix , and give a polynomial time sampling algorithm when all but eigenvalues of lie in an inte…
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
Bingbin Liu, Elan Rosenfeld, Pradeep Ravikumar +1
Noise-contrastive estimation (NCE) is a statistically consistent method for learning unnormalized probabilistic models. It has been empirically observed that the choice of the nois…
The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders
Divyansh Pareek, Andrej Risteski
Training and using modern neural-network based latent-variable generative models (like Variational Autoencoders) often require simultaneously training a generative direction along…