Reinforcement Learning with Chromatic Networks for Compact Architecture Search

Abstract

We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way. By defining the combinatorial search space of NAS to be the set of different edge-partitionings (colorings) into same-weight classes, we represent compact architectures via efficient learned edge-partitionings. For several RL tasks, we manage to learn colorings translating to effective policies parameterized by as few as $17$ weight parameters, providing >90% compression over vanilla policies and 6x compression over state-of-the-art compact policies based on Toeplitz matrices, while still maintaining good reward. We believe that our work is one of the first attempts to propose a rigorous approach to training structured neural network architectures for RL problems that are of interest especially in mobile robotics with limited storage and computational resources.

arXiv commentPublished at ICLR 2020 Neural Architecture Search Workshop. This paper is deprecated; please see arXiv:2101.07415 for the newer version

References in corpus (13)

• Natural Scales in Geographical Patterns
• Neural Architecture Search with Reinforcement Learning
• Neural Architecture Search: A Survey
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Compressing Neural Networks with the Hashing Trick
• Simple random search provides a competitive approach to reinforcement learning
• Exploring Sparsity in Recurrent Neural Networks
• Structured Transforms for Small-Footprint Deep Learning
• Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
• Exploring Randomly Wired Neural Networks for Image Recognition
• Learning Sparse Networks Using Targeted Dropout
• Policies Modulating Trajectory Generators
• Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods