VEGA: Towards an End-to-End Configurable AutoML Pipeline

Abstract

Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models. However, designing an integrated AutoML system faces four great challenges of configurability, scalability, integrability, and platform diversity. In this work, we present VEGA, an efficient and comprehensive AutoML framework that is compatible and optimized for multiple hardware platforms. a) The VEGA pipeline integrates various modules of AutoML, including Neural Architecture Search (NAS), Hyperparameter Optimization (HPO), Auto Data Augmentation, Model Compression, and Fully Train. b) To support a variety of search algorithms and tasks, we design a novel fine-grained search space and its description language to enable easy adaptation to different search algorithms and tasks. c) We abstract the common components of deep learning frameworks into a unified interface. VEGA can be executed with multiple back-ends and hardwares. Extensive benchmark experiments on multiple tasks demonstrate that VEGA can improve the existing AutoML algorithms and discover new high-performance models against SOTA methods, e.g. the searched DNet model zoo for Ascend 10x faster than EfficientNet-B5 and 9.2x faster than RegNetX-32GF on ImageNet. VEGA is open-sourced at https://github.com/huawei-noah/vega.

arXiv commentAutoML pipeline. Code is open-sourced at https://github.com/huawei-noah/vega

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• AutoML: A Survey of the State-of-the-Art
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Comparative Study of CNN and RNN for Natural Language Processing
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• AutoAugment: Learning Augmentation Policies from Data
• The EuroCity Persons Dataset: A Novel Benchmark for Object Detection
• Spatial As Deep: Spatial CNN for Traffic Scene Understanding
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
• Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
• DeepArchitect: Automatically Designing and Training Deep Architectures
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending
• Circumventing Outliers of AutoAugment with Knowledge Distillation
• An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization

Cited by in corpus (1)

• AutoML: A Survey of the State-of-the-Art