Feature Engineering for Predictive Modeling using Reinforcement Learning

Abstract

Feature engineering is a crucial step in the process of predictive modeling. It involves the transformation of given feature space, typically using mathematical functions, with the objective of reducing the modeling error for a given target. However, there is no well-defined basis for performing effective feature engineering. It involves domain knowledge, intuition, and most of all, a lengthy process of trial and error. The human attention involved in overseeing this process significantly influences the cost of model generation. We present a new framework to automate feature engineering. It is based on performance driven exploration of a transformation graph, which systematically and compactly enumerates the space of given options. A highly efficient exploration strategy is derived through reinforcement learning on past examples.

Cited by in corpus (14)

• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
• How Much Automation Does a Data Scientist Want?
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
• Techniques for Automated Machine Learning
• Engineering Reliable Deep Learning Systems
• DIFER: Differentiable Automated Feature Engineering
• DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
• SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks
• AEFE: Automatic Embedded Feature Engineering for Categorical Features
• Automating Predictive Modeling Process using Reinforcement Learning
• DarwinML: A Graph-based Evolutionary Algorithm for Automated Machine Learning
• Towards Interaction Detection Using Topological Analysis on Neural Networks
• Robusta: Robust AutoML for Feature Selection via Reinforcement Learning