Automated Machine Learning: From Principles to Practices

Abstract

Machine learning (ML) methods have been developing rapidly, but configuring and selecting proper methods to achieve a desired performance is increasingly difficult and tedious. To address this challenge, automated machine learning (AutoML) has emerged, which aims to generate satisfactory ML configurations for given tasks in a data-driven way. In this paper, we provide a comprehensive survey on this topic. We begin with the formal definition of AutoML and then introduce its principles, including the bi-level learning objective, the learning strategy, and the theoretical interpretation. Then, we summarize the AutoML practices by setting up the taxonomy of existing works based on three main factors: the search space, the search algorithm, and the evaluation strategy. Each category is also explained with the representative methods. Then, we illustrate the principles and practices with exemplary applications from configuring ML pipeline, one-shot neural architecture search, and integration with foundation models. Finally, we highlight the emerging directions of AutoML and conclude the survey.

This is a preliminary and will be kept updated

Cited by in corpus (22)

• AutoML: A Survey of the State-of-the-Art
• Concealed Object Detection
• Vertical Federated Learning: Concepts, Advances and Challenges
• Differentiable Quantum Architecture Search
• IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
• Adaptation Strategies for Automated Machine Learning on Evolving Data
• General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance
• Recommender systems based on graph embedding techniques: A comprehensive review
• Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis
• Efficient Automatic CASH via Rising Bandits
• Variational Quantum Cloning: Improving Practicality for Quantum Cryptanalysis
• OptABC: an Optimal Hyperparameter Tuning Approach for Machine Learning Algorithms
• TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning
• SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions
• Enabling Collaborative Data Science Development with the Ballet Framework
• Transfer Learning based Search Space Design for Hyperparameter Tuning
• Semantic-Based Neural Network Repair
• Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications
• Determinable and interpretable network representation for link prediction
• AutoML Meets Time Series Regression Design and Analysis of the AutoSeries Challenge
• Bridging the Gap of AutoGraph between Academia and Industry: Analysing AutoGraph Challenge at KDD Cup 2020