A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

Abstract

Nowadays, liquid rocket engines use closed-loop control at most near steady operating conditions. The control of the transient phases is traditionally performed in open-loop due to highly nonlinear system dynamics. This situation is unsatisfactory, in particular for reusable engines. The open-loop control system cannot provide optimal engine performance due to external disturbances or the degeneration of engine components over time. In this paper, we study a deep reinforcement learning approach for optimal control of a generic gas-generator engine's continuous start-up phase. It is shown that the learned policy can reach different steady-state operating points and convincingly adapt to changing system parameters. A quantitative comparison with carefully tuned open-loop sequences and PID controllers is included. The deep reinforcement learning controller achieves the highest performance and requires only minimal computational effort to calculate the control action, which is a big advantage over approaches that require online optimization, such as model predictive control. control.

References in corpus (11)

• Continuous control with deep reinforcement learning
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• Applications of Deep Learning and Reinforcement Learning to Biological Data
• Reinforcement Learning with Deep Energy-Based Policies
• Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
• Constrained Policy Optimization
• Heat Transfer Prediction for Methane in Regenerative Cooling Channels with Neural Networks
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
• Multidisciplinary Design Optimization of Reusable Launch Vehicles for Different Propellants and Objectives
• Stability-certified reinforcement learning: A control-theoretic perspective

Cited by in corpus (3)

• Early Detection of Thermoacoustic Instabilities in a Cryogenic Rocket Thrust Chamber using Combustion Noise Features and Machine Learning
• Trajectory Planning with Deep Reinforcement Learning in High-Level Action Spaces
• Machine Learning Methods for the Design and Operation of Liquid Rocket Engines -- Research Activities at the DLR Institute of Space Propulsion