Learning Event-triggered Control from Data through Joint Optimization

Abstract

We present a framework for model-free learning of event-triggered control strategies. Event-triggered methods aim to achieve high control performance while only closing the feedback loop when needed. This enables resource savings, e.g., network bandwidth if control commands are sent via communication networks, as in networked control systems. Event-triggered controllers consist of a communication policy, determining when to communicate, and a control policy, deciding what to communicate. It is essential to jointly optimize the two policies since individual optimization does not necessarily yield the overall optimal solution. To address this need for joint optimization, we propose a novel algorithm based on hierarchical reinforcement learning. The resulting algorithm is shown to accomplish high-performance control in line with resource savings and scales seamlessly to nonlinear and high-dimensional systems. The method's applicability to real-world scenarios is demonstrated through experiments on a six degrees of freedom real-time controlled manipulator. Further, we propose an approach towards evaluating the stability of the learned neural network policies.

References in corpus (11)

• Continuous control with deep reinforcement learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• End-to-End Training of Deep Visuomotor Policies
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Reinforcement Learning with Parameterized Actions
• Deep Reinforcement Learning in Parameterized Action Space
• Learnings Options End-to-End for Continuous Action Tasks
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verification
• Deep Reinforcement Learning for Event-Triggered Control
• Learning self-triggered controllers with Gaussian processes

Cited by in corpus (6)

• Deep reinforcement learning of event-triggered communication and consensus-based control for distributed cooperative transport
• A Learning Approach for Joint Design of Event-triggered Control and Power-Efficient Resource Allocation
• Experimental Study of Transport Layer Protocols for Wireless Networked Control Systems
• Safe Value Functions
• Current trends and future directions in event-based control
• Learning-based Event-triggered MPC with Gaussian processes under terminal constraints