Deep Reinforcement Learning for Six Degree-of-Freedom Planetary Powered Descent and Landing

Abstract

Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase to target specific surface locations and achieve pinpoint accuracy (landing error ellipse $<$ 5 m radius). The latter requires both a navigation system capable of estimating the lander's state in real-time and a guidance and control system that can map the estimated lander state to a commanded thrust for each lander engine. In this paper, we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The latter is used to learn a policy mapping the lander's estimated state directly to a commanded thrust for each engine, with the policy resulting in accurate and fuel-efficient trajectories. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. Another contribution of this paper is the use of different discount rates for terminal and shaping rewards, which significantly enhances optimization performance. We present simulation results demonstrating the guidance and control system's performance in a 6-DOF simulation environment and demonstrate robustness to noise and system parameter uncertainty.

37 pages

References in corpus (1)

• Constrained Policy Optimization

Cited by in corpus (10)

• Reinforcement Learning for Angle-Only Intercept Guidance of Maneuvering Targets
• Real-Time Optimal Control for Irregular Asteroid Landings Using Deep Neural Networks
• Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning
• Seeker based Adaptive Guidance via Reinforcement Meta-Learning Applied to Asteroid Close Proximity Operations
• Six Degree-of-Freedom Body-Fixed Hovering over Unmapped Asteroids via LIDAR Altimetry and Reinforcement Meta-Learning
• Reinforcement Meta-Learning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop
• Adaptive Guidance with Reinforcement Meta-Learning
• Autonomous Six-Degree-of-Freedom Spacecraft Docking Maneuvers via Reinforcement Learning
• A Survey on Artificial Intelligence Trends in Spacecraft Guidance Dynamics and Control
• Learning Accurate Extended-Horizon Predictions of High Dimensional Trajectories