617 citations · 746 across the 17 of their papers we have counts for
15 papers · 1 filter
Deep sequence models tend to memorize geometrically; it is unclear why
Shahriar Noroozizadeh, Vaishnavh Nagarajan, Elan Rosenfeld +1
Deep sequence models are said to store atomic facts predominantly in the form of associative memory: a brute-force lookup of co-occurring entities. We identify a dramatically diffe…
Hidden Breakthroughs in Language Model Training
Sara Kangaslahti, Elan Rosenfeld, Naomi Saphra
Loss curves are smooth during most of model training, so visible discontinuities stand out as possible conceptual breakthroughs. Studying these breakthroughs enables a deeper under…
Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization
Elan Rosenfeld, Andrej Risteski
We identify a new phenomenon in neural network optimization which arises from the interaction of depth and a particular heavy-tailed structure in natural data. Our result offers in…
One-Shot Strategic Classification Under Unknown Costs
Elan Rosenfeld, Nir Rosenfeld
The goal of strategic classification is to learn decision rules which are robust to strategic input manipulation. Earlier works assume that these responses are known; while some re…
Identifying Representations for Intervention Extrapolation
Sorawit Saengkyongam, Elan Rosenfeld, Pradeep Ravikumar +2
The premise of identifiable and causal representation learning is to improve the current representation learning paradigm in terms of generalizability or robustness. Despite recent…
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing
Simon Buchholz, Goutham Rajendran, Elan Rosenfeld +3
We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function i…
APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representations
Elan Rosenfeld, Preetum Nakkiran, Hadi Pouransari +2
Recent advances in learning aligned multimodal representations have been primarily driven by training large neural networks on massive, noisy paired-modality datasets. In this work…
Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization
Elan Rosenfeld, Pradeep Ravikumar, Andrej Risteski
A common explanation for the failure of deep networks to generalize out-of-distribution is that they fail to recover the "correct" features. We challenge this notion with a simple…
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
Bingbin Liu, Elan Rosenfeld, Pradeep Ravikumar +1
Noise-contrastive estimation (NCE) is a statistically consistent method for learning unnormalized probabilistic models. It has been empirically observed that the choice of the nois…
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments
Yining Chen, Elan Rosenfeld, Mark Sellke +2
Domain generalization aims at performing well on unseen test environments with data from a limited number of training environments. Despite a proliferation of proposal algorithms f…
An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization
Elan Rosenfeld, Pradeep Ravikumar, Andrej Risteski
A popular assumption for out-of-distribution generalization is that the training data comprises sub-datasets, each drawn from a distinct distribution; the goal is then to "interpol…
The Risks of Invariant Risk Minimization
Elan Rosenfeld, Pradeep Ravikumar, Andrej Risteski
Invariant Causal Prediction (Peters et al., 2016) is a technique for out-of-distribution generalization which assumes that some aspects of the data distribution vary across the tra…
Self-Reflective Variational Autoencoder
Ifigeneia Apostolopoulou, Elan Rosenfeld, Artur Dubrawski
The Variational Autoencoder (VAE) is a powerful framework for learning probabilistic latent variable generative models. However, typical assumptions on the approximate posterior di…
Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar +1
Machine learning algorithms are known to be susceptible to data poisoning attacks, where an adversary manipulates the training data to degrade performance of the resulting classifi…
Certified Adversarial Robustness via Randomized Smoothing
Jeremy M Cohen, Elan Rosenfeld, J. Zico Kolter
We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the norm.…