Penalized Partial Least Squares Based on B-Splines Transformations

Abstract

We propose a novel method to model nonlinear regression problems by adapting the principle of penalization to Partial Least Squares (PLS). Starting with a generalized additive model, we expand the additive component of each variable in terms of a generous amount of B-Splines basis functions. In order to prevent overfitting and to obtain smooth functions, we estimate the regression model by applying a penalized version of PLS. Although our motivation for penalized PLS stems from its use for B-Splines transformed data, the proposed approach is very general and can be applied to other penalty terms or to other dimension reduction techniques. It turns out that penalized PLS can be computed virtually as fast as PLS. We prove a close connection of penalized PLS to the solutions of preconditioned linear systems. In the case of high-dimensional data, the new method is shown to be an attractive competitor to other techniques for estimating generalized additive models. If the number of predictor variables is high compared to the number of examples, traditional techniques often suffer from overfitting. We illustrate that penalized PLS performs well in these situations.

Cited by in corpus (13)

• Methodology and theory for partial least squares applied to functional data
• Bayesian bandwidth estimation for a nonparametric functional regression model with mixed types of regressors and unknown error density
• On function-on-function regression: Partial least squares approach
• Sparse semiparametric regression when predictors are mixture of functional and high-dimensional variables
• Scalable Bayesian shrinkage and uncertainty quantification in high-dimensional regression
• Lanczos Approximations for the Speedup of Kernel Partial Least Squares Regression
• Bayesian Fisher's Discriminant for Functional Data
• Dual-sPLS: a family of Dual Sparse Partial Least Squares regressions for feature selection and prediction with tunable sparsity; evaluation on simulated and near-infrared (NIR) data
• Neural Networks as Functional Classifiers
• Bayesian bandwidth estimation and semi-metric selection for a functional partial linear model with unknown error density
• Sensible Functional Linear Discriminant Analysis
• Mid infrared spectroscopy and milk quality traits: a data analysis competition at the "International Workshop on Spectroscopy and Chemometrics 2021"
• The Degrees of Freedom of Partial Least Squares Regression