A hybrid learning method for system identification and optimal control

Abstract

We present a three-step method to perform system identification and optimal control of non-linear systems. Our approach is mainly data driven and does not require active excitation of the system to perform system identification. In particular, it is designed for systems for which only historical data under closed-loop control are available and where historical control commands exhibit low variability. In a first step, simple simulation models of the system are built and run under various conditions. In a second step, a neural network architecture is extensively trained on the simulation outputs to learn the system physics, and retrained with historical data from the real system with stopping rules. These constraints avoid overfitting that arise by fitting closed-loop controlled systems. By doing so, we obtain one (or many) system model(s), represented by this architecture, and whose behaviour can be chosen to match more or less the real system. Finally, state-of-the-art reinforcement learning with a variant of domain randomization and distributed learning is used for optimal control of the system. We first illustrate the model identification strategy with a simple example, the pendulum with external torque. We then apply our method to model and optimize the control of a large building facility located in Switzerland. Simulation results demonstrate that this approach generates stable functional controllers which outperform on comfort and energy benchmark rule-based controllers.

arXiv comment15 pages, Accepted for publication in TNNLS

References in corpus (14)

• Sequence to Sequence Learning with Neural Networks
• Trust Region Policy Optimization
• Data-driven discovery of coordinates and governing equations
• Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
• Emergence of Locomotion Behaviours in Rich Environments
• An Algorithmic Perspective on Imitation Learning
• Learning Dexterous In-Hand Manipulation
• Deep Learning and Model Predictive Control for Self-Tuning Mode-Locked Lasers
• SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
• Linear System Identification in a Nonlinear Setting - Nonparametric analysis of the nonlinear distortions and their impact on the best linear approximation
• Nonlinear Systems Identification Using Deep Dynamic Neural Networks
• Why gradient clipping accelerates training: A theoretical justification for adaptivity
• Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy
• Learning latent state representation for speeding up exploration