Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search

Abstract

Fabricating neural models for a wide range of mobile devices demands for a specific design of networks due to highly constrained resources. Both evolution algorithms (EA) and reinforced learning methods (RL) have been dedicated to solve neural architecture search problems. However, these combinations usually concentrate on a single objective such as the error rate of image classification. They also fail to harness the very benefits from both sides. In this paper, we present a new multi-objective oriented algorithm called MoreMNAS (Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search) by leveraging good virtues from both EA and RL. In particular, we incorporate a variant of multi-objective genetic algorithm NSGA-II, in which the search space is composed of various cells so that crossovers and mutations can be performed at the cell level. Moreover, reinforced control is mixed with a natural mutating process to regulate arbitrary mutation, maintaining a delicate balance between exploration and exploitation. Therefore, not only does our method prevent the searched models from degrading during the evolution process, but it also makes better use of learned knowledge. Our experiments conducted in Super-resolution domain (SR) deliver rivalling models compared to some state-of-the-art methods with fewer FLOPS.

Deep Learning, Neural Architecture Search, Multi-objective, Reinforcement Learning

References in corpus (13)

• Neural Architecture Search with Reinforcement Learning
• Roulette-wheel selection via stochastic acceptance
• Efficient Neural Architecture Search via Parameter Sharing
• Designing Neural Network Architectures using Reinforcement Learning
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Hierarchical Representations for Efficient Architecture Search
• Progressive Neural Architecture Search
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
• DeepArchitect: Automatically Designing and Training Deep Architectures
• NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
• Reinforced Evolutionary Neural Architecture Search
• Improved Crowding Distance for NSGA-II
• Policy Optimization With Penalized Point Probability Distance: An Alternative To Proximal Policy Optimization

Cited by in corpus (14)

• A Survey on Evolutionary Neural Architecture Search
• Object Detection in 20 Years: A Survey
• Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search
• Lightweight Image Super-Resolution with Adaptive Weighted Learning Network
• A Comprehensive Survey on Hardware-Aware Neural Architecture Search
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Lightweight Single-Image Super-Resolution Network with Attentive Auxiliary Feature Learning
• GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
• Collapsible Linear Blocks for Super-Efficient Super Resolution
• Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
• MoGA: Searching Beyond MobileNetV3
• A Matrix-in-matrix Neural Network for Image Super Resolution
• Neural Architecture Search for Image Super-Resolution Using Densely Constructed Search Space: DeCoNAS
• Memory-Efficient Hierarchical Neural Architecture Search for Image Restoration