Grouped variable importance with random forests and application to multiple functional data analysis

Abstract

The selection of grouped variables using the random forest algorithm is considered. First a new importance measure adapted for groups of variables is proposed. Theoretical insights into this criterion are given for additive regression models. Second, an original method for selecting functional variables based on the grouped variable importance measure is developed. Using a wavelet basis, it is proposed to regroup all of the wavelet coefficients for a given functional variable and use a wrapper selection algorithm with these groups. Various other groupings which take advantage of the frequency and time localization of the wavelet basis are proposed. An extensive simulation study is performed to illustrate the use of the grouped importance measure in this context. The method is applied to a real life problem coming from aviation safety.

References in corpus (6)

• Correlation and variable importance in random forests
• Prediction in functional linear regression
• Support vector machine for functional data classification
• Functional additive regression
• Variable selection for the multicategory SVM via adaptive sup-norm regularization
• Classification supervisée en grande dimension. Application à l'agrément de conduite automobile

Cited by in corpus (9)

• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Grouped Feature Importance and Combined Features Effect Plot
• Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams
• Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance
• Functional archetype and archetypoid analysis
• Interpreting Deep Neural Networks Through Variable Importance
• Triplot: model agnostic measures and visualisations for variable importance in predictive models that take into account the hierarchical correlation structure
• Supervised Learning via Ensembles of Diverse Functional Representations: the Functional Voting Classifier
• A statistical machine learning approach for benchmarking in the presence of complex contextual factors and peer groups