Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science

Abstract

As the field of data science continues to grow, there will be an ever-increasing demand for tools that make machine learning accessible to non-experts. In this paper, we introduce the concept of tree-based pipeline optimization for automating one of the most tedious parts of machine learning---pipeline design. We implement an open source Tree-based Pipeline Optimization Tool (TPOT) in Python and demonstrate its effectiveness on a series of simulated and real-world benchmark data sets. In particular, we show that TPOT can design machine learning pipelines that provide a significant improvement over a basic machine learning analysis while requiring little to no input nor prior knowledge from the user. We also address the tendency for TPOT to design overly complex pipelines by integrating Pareto optimization, which produces compact pipelines without sacrificing classification accuracy. As such, this work represents an important step toward fully automating machine learning pipeline design.

8 pages, 5 figures, preprint to appear in GECCO 2016, edits not yet made from reviewer comments

References in corpus (3)

• Scikit-learn: Machine Learning in Python
• Practical Bayesian Optimization of Machine Learning Algorithms
• Automating biomedical data science through tree-based pipeline optimization

Cited by in corpus (6)

• Autostacker: A Compositional Evolutionary Learning System
• Automated Machine Learning Techniques for Data Streams
• Automated Machine Learning Service Composition
• Toward the automated analysis of complex diseases in genome-wide association studies using genetic programming
• Quantifying error contributions of computational steps, algorithms and hyperparameter choices in image classification pipelines
• An Extensive Experimental Evaluation of Automated Machine Learning Methods for Recommending Classification Algorithms (Extended Version)