Control of a Quadrotor with Reinforcement Learning

Abstract

In this paper, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training. Moreover, we present a new learning algorithm which differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. We found that it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside-down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7 μs per time step which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.

Cited by in corpus (34)

• Learning agile and dynamic motor skills for legged robots
• Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning
• Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning
• Neural Network-based Flight Control Systems: Present and Future
• Driving in Dense Traffic with Model-Free Reinforcement Learning
• Learning Minimum-Time Flight in Cluttered Environments
• Real-time System Identification Using Deep Learning for Linear Processes with Application to Unmanned Aerial Vehicles
• Build Your Own Visual-Inertial Drone: A Cost-Effective and Open-Source Autonomous Drone
• Inclined Quadrotor Landing using Deep Reinforcement Learning
• Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Control
• Learning High-Level Policies for Model Predictive Control
• End-to-end Neural Network Based Quadcopter control
• Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control
• D-VAT: End-to-End Visual Active Tracking for Micro Aerial Vehicles
• Aggressive Online Control of a Quadrotor via Deep Network Representations of Optimality Principles
• Machine learning and control engineering: The model-free case
• Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs
• Navigating Assistance System for Quadcopter with Deep Reinforcement Learning
• End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight
• DeepSoCS: A Neural Scheduler for Heterogeneous System-on-Chip (SoC) Resource Scheduling
• A Learning-based Quadcopter Controller with Extreme Adaptation
• Learning Quadrotor Control From Visual Features Using Differentiable Simulation
• The Power of Input: Benchmarking Zero-Shot Sim-To-Real Transfer of Reinforcement Learning Control Policies for Quadrotor Control
• Learning Autonomous Race Driving with Action Mapping Reinforcement Learning
• Exploring Deep Reinforcement Learning for Robust Target Tracking using Micro Aerial Vehicles
• Neuromorphic computing for attitude estimation onboard quadrotors
• Aerial Field Robotics
• Control of a Twin Rotor using Twin Delayed Deep Deterministic Policy Gradient (TD3)
• Autonomous Curiosity for Real-Time Training Onboard Robotic Agents
• Fault Tolerant Control of a Quadcopter using Reinforcement Learning
• Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control
• How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learning
• Dynamics-Invariant Quadrotor Control using Scale-Aware Deep Reinforcement Learning
• High-Performance Model Predictive Control for Quadcopters with Formal Stability Guarantees