Population Based Training of Neural Networks

Abstract

Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we present \emph{Population Based Training (PBT)}, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance. Importantly, PBT discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training. With just a small modification to a typical distributed hyperparameter training framework, our method allows robust and reliable training of models. We demonstrate the effectiveness of PBT on deep reinforcement learning problems, showing faster wall-clock convergence and higher final performance of agents by optimising over a suite of hyperparameters. In addition, we show the same method can be applied to supervised learning for machine translation, where PBT is used to maximise the BLEU score directly, and also to training of Generative Adversarial Networks to maximise the Inception score of generated images. In all cases PBT results in the automatic discovery of hyperparameter schedules and model selection which results in stable training and better final performance.

Cited by in corpus (29)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• Automated Machine Learning in Practice: State of the Art and Recent Results
• Population-Based Black-Box Optimization for Biological Sequence Design
• Evolutionary Reinforcement Learning: A Survey
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• An overview of mixing augmentation methods and augmentation strategies
• Fast simulation of muons produced at the SHiP experiment using Generative Adversarial Networks
• Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking
• Neural Networks Versus Conventional Filters for Inertial-Sensor-based Attitude Estimation
• HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents
• Meta-Learning by the Baldwin Effect
• Representation learning for neural population activity with Neural Data Transformers
• A Reinforcement Learning Approach for the Continuous Electricity Market of Germany: Trading from the Perspective of a Wind Park Operator
• Forward and inverse reinforcement learning sharing network weights and hyperparameters
• EvoJAX: Hardware-Accelerated Neuroevolution
• PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds
• HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization
• Emulation Techniques for Scenario and Classical Control Design of Tokamak Plasmas
• Distributed Evolution Strategies with Multi-Level Learning for Large-Scale Black-Box Optimization
• HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach
• Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations
• Distributed Training and Optimization Of Neural Networks
• Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation
• BoGraph: Structured Bayesian Optimization From Logs for Expensive Systems with Many Parameters
• Optimizing Neural Network Scale for ECG Classification
• Resource-Adaptive Successive Doubling for Hyperparameter Optimization with Large Datasets on High-Performance Computing Systems
• LA3: Efficient Label-Aware AutoAugment