Automated Machine Learning: State-of-The-Art and Open Challenges

Abstract

With the continuous and vast increase in the amount of data in our digital world, it has been acknowledged that the number of knowledgeable data scientists can not scale to address these challenges. Thus, there was a crucial need for automating the process of building good machine learning models. In the last few years, several techniques and frameworks have been introduced to tackle the challenge of automating the process of Combined Algorithm Selection and Hyper-parameter tuning (CASH) in the machine learning domain. The main aim of these techniques is to reduce the role of the human in the loop and fill the gap for non-expert machine learning users by playing the role of the domain expert. In this paper, we present a comprehensive survey for the state-of-the-art efforts in tackling the CASH problem. In addition, we highlight the research work of automating the other steps of the full complex machine learning pipeline (AutoML) from data understanding till model deployment. Furthermore, we provide comprehensive coverage for the various tools and frameworks that have been introduced in this domain. Finally, we discuss some of the research directions and open challenges that need to be addressed in order to achieve the vision and goals of the AutoML process.

References in corpus (10)

• Practical Bayesian Optimization of Machine Learning Algorithms
• ADADELTA: An Adaptive Learning Rate Method
• Neural Architecture Search with Reinforcement Learning
• How transferable are features in deep neural networks?
• OpenML: networked science in machine learning
• Scalable Bayesian Optimization Using Deep Neural Networks
• Efficient Architecture Search by Network Transformation
• BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
• Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
• Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start

Cited by in corpus (7)

• Neural Architecture Search For Fault Diagnosis
• AutoAI-TS: AutoAI for Time Series Forecasting
• Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning
• AutoML: Exploration v.s. Exploitation
• Automated Machine Learning Techniques for Data Streams
• Enabling SQL-based Training Data Debugging for Federated Learning
• Prediction of properties of metal alloy materials based on machine learning