7 papers · 1 filter
SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model
Tyler LaBonte, Vidya Muthukumar
Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully underst…
How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
Kuo-Wei Lai, Guanghui Wang, Molei Tao +1
Overparameterized ML models, including neural networks, typically induce underdetermined training objectives with multiple global minima. The implicit bias refers to the limiting g…
A general technique for approximating high-dimensional empirical kernel matrices
Chiraag Kaushik, Justin Romberg, Vidya Muthukumar
We present simple, user-friendly bounds for the expected operator norm of a random kernel matrix under general conditions on the kernel function . Our approach uses…
General Loss Functions Lead to (Approximate) Interpolation in High Dimensions
Kuo-Wei Lai, Vidya Muthukumar
We provide a unified framework that applies to a general family of convex losses across binary and multiclass settings in the overparameterized regime to approximately characterize…
Estimating stationary mass, frequency by frequency
Milind Nakul, Vidya Muthukumar, Ashwin Pananjady
Suppose we observe a trajectory of length from an exponentially -mixing stochastic process over a finite but potentially large state space. We consider the problem of estim…
Task Shift: From Classification to Regression in Overparameterized Linear Models
Tyler LaBonte, Kuo-Wei Lai, Vidya Muthukumar
Modern machine learning methods have recently demonstrated remarkable capability to generalize under task shift, where latent knowledge is transferred to a different, often more di…