Reliability Quantification of Deep Reinforcement Learning-based Control

Abstract

Reliability quantification of deep reinforcement learning (DRL)-based control is a significant challenge for the practical application of artificial intelligence (AI) in safety-critical systems. This study proposes a method for quantifying the reliability of DRL-based control. First, an existing method, random noise distillation, was applied to the reliability evaluation to clarify the issues to be solved. Second, a novel method for reliability quantification was proposed to solve these issues. The reliability is quantified using two neural networks: reference and evaluator. They have the same structure with the same initial parameters. The outputs of the two networks were the same before training. During training, the evaluator network parameters were updated to maximize the difference between the reference and evaluator networks for trained data. Thus, the reliability of the DRL-based control for a state can be evaluated based on the difference in output between the two networks. The proposed method was applied to DQN-based control as an example of a simple task, and its effectiveness was demonstrated. Finally, the proposed method was applied to the problem of switching trained models depending on the state. Con-sequently, the performance of the DRL-based control was improved by switching the trained models according to their reliability.

arXiv comment18 pages and 17 figures

References in corpus (9)

• Continuous control with deep reinforcement learning
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Predictive Uncertainty Estimation via Prior Networks
• High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
• BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning
• Uncertainty-Based Out-of-Distribution Classification in Deep Reinforcement Learning
• A general framework for ensemble distribution distillation
• Out-of-distribution detection in satellite image classification