On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions

Denys Pushkin, Raphaël Berthier, Emmanuel Abbe

We investigate the out-of-domain generalization of random feature (RF) models and Transformers. We first prove that in the `generalization on the unseen (GOTU)' setting, where trai…