The NN-Stacking: Feature weighted linear stacking through neural networks
arXiv:1906.09735 · doi:10.1016/j.neucom.2020.02.073
Abstract
Stacking methods improve the prediction performance of regression models. A simple way to stack base regressions estimators is by combining them linearly, as done by \citet{breiman1996stacked}. Even though this approach is useful from an interpretative perspective, it often does not lead to high predictive power. We propose the NN-Stacking method (NNS), which generalizes Breiman's method by allowing the linear parameters to vary with input features. This improvement enables NNS to take advantage of the fact that distinct base models often perform better at different regions of the feature space. Our method uses neural networks to estimate the stacking coefficients. We show that while our approach keeps the interpretative features of Breiman's method at a local level, it leads to better predictive power, especially in datasets with large sample sizes.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Scikit-learn: Machine Learning in Python
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Improving neural networks by preventing co-adaptation of feature detectors
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Feature-Weighted Linear Stacking
- A Unified Framework for Constructing, Tuning and Assessing Photometric Redshift Density Estimates in a Selection Bias Setting
- A Data-Driven Statistical Model for Predicting the Critical Temperature of a Superconductor
- NLS: an accurate and yet easy-to-interpret regression method