ns3-gym: Extending OpenAI Gym for Networking Research

Abstract

OpenAI Gym is a toolkit for reinforcement learning (RL) research. It includes a large number of well-known problems that expose a common interface allowing to directly compare the performance results of different RL algorithms. Since many years, the ns-3 network simulation tool is the de-facto standard for academic and industry research into networking protocols and communications technology. Numerous scientific papers were written reporting results obtained using ns-3, and hundreds of models and modules were written and contributed to the ns-3 code base. Today as a major trend in network research we see the use of machine learning tools like RL. What is missing is the integration of a RL framework like OpenAI Gym into the network simulator ns-3. This paper presents the ns3-gym framework. First, we discuss design decisions that went into the software. Second, two illustrative examples implemented using ns3-gym are presented. Our software package is provided to the community as open source under a GPL license and hence can be easily extended.

References in corpus (1)

• Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo

Cited by in corpus (8)

• Deep Learning in Mobile and Wireless Networking: A Survey
• QoS-Aware Load Balancing in Wireless Networks using Clipped Double Q-Learning
• PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems
• The Adversarial Resilience Learning Architecture for AI-based Modelling, Exploration, and Operation of Complex Cyber-Physical Systems
• When Machine Learning Meets Congestion Control: A Survey and Comparison
• Komondor: a Wireless Network Simulator for Next-Generation High-Density WLANs
• Towards Cognitive Routing based on Deep Reinforcement Learning
• Wield: Systematic Reinforcement Learning With Progressive Randomization