Whitening Improves Robustness to Spurious Correlations in Linear Probes

Floris Holstege, Bram Wouters, Noud van Giersbergen +1

Deep neural networks tend to rely on simple features that may be spurious and thus fail to generalize. We study this problem in the setting of linear probes, where a (generalized)…