When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

Abstract

This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when the model is nonparametric and well-specified. We examine the missing case in the literature where the model is nonparametric and misspecified, and show that the IW correction is needed for obtaining the best approximation of the true unknown function for the test distribution. We do this by analyzing IW-corrected kernel ridge regression, covering a variety of settings, including parametric and nonparametric models, well-specified and misspecified settings, and arbitrary weighting functions.

References in corpus (10)

• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Domain Adaptation: Learning Bounds and Algorithms
• What is the Effect of Importance Weighting in Deep Learning?
• Generalization Bounds for Domain Adaptation
• Understanding the role of importance weighting for deep learning
• A new similarity measure for covariate shift with applications to nonparametric regression
• Understanding Why Generalized Reweighting Does Not Improve Over ERM
• Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift
• Is Importance Weighting Incompatible with Interpolating Classifiers?
• Local convergence rates of the nonparametric least squares estimator with applications to transfer learning