D2RL: Deep Dense Architectures in Reinforcement Learning

Abstract

While improvements in deep learning architectures have played a crucial role in improving the state of supervised and unsupervised learning in computer vision and natural language processing, neural network architecture choices for reinforcement learning remain relatively under-explored. We take inspiration from successful architectural choices in computer vision and generative modelling, and investigate the use of deeper networks and dense connections for reinforcement learning on a variety of simulated robotic learning benchmark environments. Our findings reveal that current methods benefit significantly from dense connections and deeper networks, across a suite of manipulation and locomotion tasks, for both proprioceptive and image-based observations. We hope that our results can serve as a strong baseline and further motivate future research into neural network architectures for reinforcement learning. The project website with code is at this link https://sites.google.com/view/d2rl/home.

NeurIPS 2020 Deep RL Workshop. Preprint. Under review

References in corpus (18)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Simple Framework for Contrastive Learning of Visual Representations
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Solving Rubik's Cube with a Robot Hand
• Challenges of Real-World Reinforcement Learning
• Data-Efficient Hierarchical Reinforcement Learning
• Reinforcement Learning with Augmented Data
• Dream to Control: Learning Behaviors by Latent Imagination
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• The power of deeper networks for expressing natural functions
• An empirical investigation of the challenges of real-world reinforcement learning
• Observational Overfitting in Reinforcement Learning
• Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?
• Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning

Cited by in corpus (8)

• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• On the model-based stochastic value gradient for continuous reinforcement learning
• Training Larger Networks for Deep Reinforcement Learning
• S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning
• Towards Deeper Deep Reinforcement Learning with Spectral Normalization
• Recomposing the Reinforcement Learning Building Blocks with Hypernetworks
• Hierarchical Skills for Efficient Exploration