Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
arXiv:2106.11890
Abstract
When tuning the architecture and hyperparameters of large machine learning models for on-device deployment, it is desirable to understand the optimal trade-offs between on-device latency and model accuracy. In this work, we leverage recent methodological advances in Bayesian optimization over high-dimensional search spaces and multi-objective Bayesian optimization to efficiently explore these trade-offs for a production-scale on-device natural language understanding model at Facebook.
arXiv commentTo Appear at the 8th ICML Workshop on Automated Machine Learning, ICML 2021
References in corpus (8)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Neural Architecture Search with Reinforcement Learning
- A Tutorial on Bayesian Optimization
- Neural Architecture Search with Bayesian Optimisation and Optimal Transport
- Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
- Predictive Entropy Search for Multi-objective Bayesian Optimization
- Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement
- AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling