Random Error Sampling-based Recurrent Neural Network Architecture Optimization

Abstract

Recurrent neural networks are good at solving prediction problems. However, finding a network that suits a problem is quite hard because their performance is strongly affected by their architecture configuration. Automatic architecture optimization methods help to find the most suitable design, but they are not extensively adopted because of their high computational cost. In this work, we introduce the Random Error Sampling-based Neuroevolution (RESN), an evolutionary algorithm that uses the mean absolute error random sampling, a training-free approach to predict the expected performance of an artificial neural network, to optimize the architecture of a network. We empirically validate our proposal on three prediction problems, and compare our technique to training-based architecture optimization techniques and to neuroevolutionary approaches. Our findings show that we can achieve state-of-the-art error performance and that we reduce by half the time needed to perform the optimization.

References in corpus (7)

• On the difficulty of training Recurrent Neural Networks
• Metaheuristic Design of Feedforward Neural Networks: A Review of Two Decades of Research
• Metaheuristic Algorithms for Convolution Neural Network
• Spatial Evolutionary Generative Adversarial Networks
• Infrastructure Deployment in Vehicular Communication Networks Using a Parallel Multiobjective Evolutionary Algorithm
• Parallel multi-objective metaheuristics for smart communications in vehicular networks
• Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization

Cited by in corpus (6)

• A Survey on Evolutionary Neural Architecture Search
• Bayesian Neural Architecture Search using A Training-Free Performance Metric
• The Ant Swarm Neuro-Evolution Procedure for Optimizing Recurrent Networks
• Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems
• Reliable and Fast Recurrent Neural Network Architecture Optimization
• An Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution