Network Environment Design for Autonomous Cyberdefense

Abstract

Reinforcement learning (RL) has been demonstrated suitable to develop agents that play complex games with human-level performance. However, it is not understood how to effectively use RL to perform cybersecurity tasks. To develop such understanding, it is necessary to develop RL agents using simulation and emulation systems allowing researchers to model a broad class of realistic threats and network conditions. Demonstrating that a specific RL algorithm can be effective for defending a network under certain conditions may not necessarily give insight about the performance of the algorithm when the threats, network conditions, and security goals change. This paper introduces a novel approach for network environment design and a software framework to address the fundamental problem that network defense cannot be defined as a single game with a simple set of fixed rules. We show how our approach is necessary to facilitate the development of RL network defenders that are robust against attacks aimed at the agent's learning. Our framework enables the development and simulation of adversaries with sophisticated behavior that includes poisoning and evasion attacks on RL network defenders.

References in corpus (11)

• Playing Atari with Deep Reinforcement Learning
• StarCraft II: A New Challenge for Reinforcement Learning
• Solving Rubik's Cube with a Robot Hand
• Delving into adversarial attacks on deep policies
• Adversarial Policies: Attacking Deep Reinforcement Learning
• Characterizing Attacks on Deep Reinforcement Learning
• CybORG: An Autonomous Cyber Operations Research Gym
• Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control
• Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence

Cited by in corpus (6)

• Learning Near-Optimal Intrusion Responses Against Dynamic Attackers
• Intrusion Prevention through Optimal Stopping
• Deception for Cyber Defence: Challenges and Opportunities
• CybORG: A Gym for the Development of Autonomous Cyber Agents
• Network Defense is Not a Game
• Modelling Direct Messaging Networks with Multiple Recipients for Cyber Deception