Input-Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance

Samet Demir, Zafer Dogan

Random feature models (RFMs), two-layer networks with a randomly initialized fixed first layer and a trained linear readout, are among the simplest nonlinear predictors. Prior asym…