16 citations · 79 across the 28 of their papers we have counts for
14 papers · 1 filter
Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task
Alicia Curth, Rachel Lawrence, Sushrut Karmalkar +1
We investigate whether transformers use their depth adaptively across tasks of increasing difficulty. Using a controlled multi-hop relational reasoning task based on family stories…
Robust Learning of a Group DRO Neuron
Guyang Cao, Shuyao Li, Sushrut Karmalkar +1
We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of c…
Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
Iskander Azangulov, Teodora Pandeva, Niranjani Prasad +2
Masked diffusion models (MDMs) offer a compelling alternative to autoregressive models (ARMs) for discrete text generation because they enable parallel token sampling, rather than…
On Learning Parallel Pancakes with Mostly Uniform Weights
Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar +2
We study the complexity of learning -mixtures of Gaussians (-GMMs) on . This task is known to have complexity in full generality. To circumvent this…
Efficient Knowledge Distillation via Curriculum Extraction
Shivam Gupta, Sushrut Karmalkar
Knowledge distillation is a technique used to train a small student network using the output generated by a large teacher network, and has many empirical advantages~\citep{Hinton20…
Batch List-Decodable Linear Regression via Higher Moments
Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar +2
We study the task of list-decodable linear regression using batches. A batch is called clean if it consists of i.i.d. samples from an unknown linear regression distribution. For a…
Learning a Single Neuron Robustly to Distributional Shifts and Adversarial Label Noise
Shuyao Li, Sushrut Karmalkar, Ilias Diakonikolas +1
We study the problem of learning a single neuron with respect to the -loss in the presence of adversarial distribution shifts, where the labels can be arbitrary, and the goa…
Sum-of-squares lower bounds for Non-Gaussian Component Analysis
Ilias Diakonikolas, Sushrut Karmalkar, Shuo Pang +1
Non-Gaussian Component Analysis (NGCA) is the statistical task of finding a non-Gaussian direction in a high-dimensional dataset. Specifically, given i.i.d.\ samples from a distrib…
Robust Sparse Estimation for Gaussians with Optimal Error under Huber Contamination
Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar +2
We study Gaussian sparse estimation tasks in Huber's contamination model with a focus on mean estimation, PCA, and linear regression. For each of these tasks, we give the first sam…
Fairness for Image Generation with Uncertain Sensitive Attributes
Ajil Jalal, Sushrut Karmalkar, Jessica Hoffmann +2
This work tackles the issue of fairness in the context of generative procedures, such as image super-resolution, which entail different definitions from the standard classification…
Instance-Optimal Compressed Sensing via Posterior Sampling
Ajil Jalal, Sushrut Karmalkar, Alexandros G. Dimakis +1
We characterize the measurement complexity of compressed sensing of signals drawn from a known prior distribution, even when the support of the prior is the entire space (rather th…
Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent
Surbhi Goel, Aravind Gollakota, Zhihan Jin +2
We prove the first superpolynomial lower bounds for learning one-layer neural networks with respect to the Gaussian distribution using gradient descent. We show that any classifier…
Approximation Schemes for ReLU Regression
Ilias Diakonikolas, Surbhi Goel, Sushrut Karmalkar +2
We consider the fundamental problem of ReLU regression, where the goal is to output the best fitting ReLU with respect to square loss given access to draws from some unknown distri…
Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
Surbhi Goel, Sushrut Karmalkar, Adam Klivans
We consider the problem of computing the best-fitting ReLU with respect to square-loss on a training set when the examples have been drawn according to a spherical Gaussian distrib…