Spectral Overfitting in Noisy Linear Probing of Pretrained Representations

Zice Wang, Zhenyu Zhang

Frozen pretrained features are often treated as a safe interface for downstream learning: only a small linear readout is trained, while the backbone is fixed. We show that this rea…