Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning

Abstract

With the rapid development of software and distributed computing, Cyber-Physical Systems (CPS) are widely adopted in many application areas, e.g., smart grid, autonomous automobile. It is difficult to detect defects in CPS models due to the complexities involved in the software and physical systems. To find defects in CPS models efficiently, robustness guided falsification of CPS is introduced. Existing methods use several optimization techniques to generate counterexamples, which falsify the given properties of a CPS. However those methods may require a large number of simulation runs to find the counterexample and is far from practical. In this work, we explore state-of-the-art Deep Reinforcement Learning (DRL) techniques to reduce the number of simulation runs required to find such counterexamples. We report our method and the preliminary evaluation results.

arXiv comment9 pages, 1 figure, to be presented at FM2018

References in corpus (3)

• Continuous Deep Q-Learning with Model-based Acceleration
• On the Robustness of Temporal Properties for Stochastic Models
• An Active Learning Approach to the Falsification of Black Box Cyber-Physical Systems

Cited by in corpus (11)

• Deep Reinforcement Learning for Cyber Security
• A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems
• Verifiably Safe Off-Model Reinforcement Learning
• Falsification of Cyber-Physical Systems with Robustness-Guided Black-Box Checking
• Active Fuzzing for Testing and Securing Cyber-Physical Systems
• When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
• Approximation-Refinement Testing of Compute-Intensive Cyber-Physical Models: An Approach Based on System Identification
• Automatic Testing With Reusable Adversarial Agents
• Multi-Armed Bandits for Boolean Connectives in Hybrid System Falsification (Extended Version)
• Worrisome Properties of Neural Network Controllers and Their Symbolic Representations
• Constraining Counterexamples in Hybrid System Falsification: Penalty-Based Approaches