DOB-Net: Actively Rejecting Unknown Excessive Time-Varying Disturbances

Abstract

This paper presents an observer-integrated Reinforcement Learning (RL) approach, called Disturbance OBserver Network (DOB-Net), for robots operating in environments where disturbances are unknown and time-varying, and may frequently exceed robot control capabilities. The DOB-Net integrates a disturbance dynamics observer network and a controller network. Originated from conventional DOB mechanisms, the observer is built and enhanced via Recurrent Neural Networks (RNNs), encoding estimation of past values and prediction of future values of unknown disturbances in RNN hidden state. Such encoding allows the controller generate optimal control signals to actively reject disturbances, under the constraints of robot control capabilities. The observer and the controller are jointly learned within policy optimization by advantage actor critic. Numerical simulations on position regulation tasks have demonstrated that the proposed DOB-Net significantly outperforms a conventional feedback controller and classical RL algorithms.

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Trust Region Policy Optimization
• Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
• Meta Reinforcement Learning with Latent Variable Gaussian Processes
• Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
• Particle Filter Networks with Application to Visual Localization

Cited by in corpus (2)

• A2: Extracting Cyclic Switchings from DOB-nets for Rejecting Excessive Disturbances
• Including Image-based Perception in Disturbance Observer for Warehouse Drones