Reinforcement Learning for Industrial Control Network Cyber Security Orchestration

Abstract

Defending computer networks from cyber attack requires coordinating actions across multiple nodes based on imperfect indicators of compromise while minimizing disruptions to network operations. Advanced attacks can progress with few observable signals over several months before execution. The resulting sequential decision problem has large observation and action spaces and a long time-horizon, making it difficult to solve with existing methods. In this work, we present techniques to scale deep reinforcement learning to solve the cyber security orchestration problem for large industrial control networks. We propose a novel attention-based neural architecture with size complexity that is invariant to the size of the network under protection. A pre-training curriculum is presented to overcome early exploration difficulty. Experiments show in that the proposed approaches greatly improve both the learning sample complexity and converged policy performance over baseline methods in simulation.

arXiv comment12 pages, submitted to NeurIPS 2021

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Playing Atari with Deep Reinforcement Learning
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Deep Reinforcement Learning for Cyber Security
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Hindsight Experience Replay
• Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving
• Deep Q-learning from Demonstrations
• Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
• Phasic Policy Gradient
• Adversarial Reinforcement Learning for Observer Design in Autonomous Systems under Cyber Attacks