CARLA: An Open Urban Driving Simulator
arXiv:1711.03938
Abstract
We introduce CARLA, an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites and environmental conditions. We use CARLA to study the performance of three approaches to autonomous driving: a classic modular pipeline, an end-to-end model trained via imitation learning, and an end-to-end model trained via reinforcement learning. The approaches are evaluated in controlled scenarios of increasing difficulty, and their performance is examined via metrics provided by CARLA, illustrating the platform's utility for autonomous driving research. The supplementary video can be viewed at https://youtu.be/Hp8Dz-Zek2E
arXiv commentPublished at the 1st Conference on Robot Learning (CoRL)
References in corpus (3)
Cited by in corpus (123)
- Shortcut Learning in Deep Neural Networks
- On Evaluation of Embodied Navigation Agents
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
- Cross-view Semantic Segmentation for Sensing Surroundings
- A Survey of End-to-End Driving: Architectures and Training Methods
- A Survey of Deep RL and IL for Autonomous Driving Policy Learning
- AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms
- Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding
- Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
- Argoverse: 3D Tracking and Forecasting with Rich Maps
- MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
- SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
- Decoupling Representation Learning from Reinforcement Learning
- Testing and verification of neural-network-based safety-critical control software: A systematic literature review
- Efficient Black-box Assessment of Autonomous Vehicle Safety
- Joint Monocular 3D Vehicle Detection and Tracking
- The StreetLearn Environment and Dataset
- Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation
- A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
- Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
- SceML - A Graphical Modeling Framework for Scenario-based Testing of Autonomous Vehicles
- Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability
- Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety
- Model-free Deep Reinforcement Learning for Urban Autonomous Driving
- Overview of Tools Supporting Planning for Automated Driving
- MADRaS : Multi Agent Driving Simulator
- Unsupervised Domain Adaptation in Semantic Segmentation: a Review
- Computing Systems for Autonomous Driving: State-of-the-Art and Challenges
- Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning
- Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey
- Motion Estimation of Connected and Automated Vehicles under Communication Delay and Packet Loss of V2X Communications
- Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
- Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing
- Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments
- A Game Theory Based Ramp Merging Strategy for Connected and Automated Vehicles in the Mixed Traffic: A Unity-SUMO Integrated Platform
- VirtualHome: Simulating Household Activities via Programs
- The State of Modeling, Simulation, and Data Utilization within Industry: An Autonomous Vehicles Perspective
- ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllers
- CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
- AutoRVO: Local Navigation with Dynamic Constraints in Dense Heterogeneous Traffic
- SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
- 3DB: A Framework for Debugging Computer Vision Models
- Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
- A Little Fog for a Large Turn
- Learning Latent Representations to Influence Multi-Agent Interaction
- Learning To Simulate
- DSDNet: Deep Structured self-Driving Network
- The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
- Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles
- Digital Twin-Assisted Cooperative Driving at Non-Signalized Intersections
- Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
- Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
- End-to-end driving simulation via angle branched network
- Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation
- DeepRacing: Parameterized Trajectories for Autonomous Racing
- Affordance Learning In Direct Perception for Autonomous Driving
- SUMMIT: A Simulator for Urban Driving in Massive Mixed Traffic
- OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research
- Feature Sharing and Integration for Cooperative Cognition and Perception with Volumetric Sensors
- Learning hierarchical behavior and motion planning for autonomous driving
- BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
- Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving
- Domain-invariant Stereo Matching Networks
- Recovering and Simulating Pedestrians in the Wild
- Attention-Privileged Reinforcement Learning
- Shadow Transfer: Single Image Relighting For Urban Road Scenes
- Semantic Road Layout Understanding by Generative Adversarial Inpainting
- Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments
- Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs
- Dynamic Objects Segmentation for Visual Localization in Urban Environments
- Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars
- On Machine Learning and Structure for Mobile Robots
- Pedestrian Collision Avoidance for Autonomous Vehicles at Unsignalized Intersection Using Deep Q-Network
- Augmented Reality-Based Advanced Driver-Assistance System for Connected Vehicles
- Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
- SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving
- AutoFlow: Learning a Better Training Set for Optical Flow
- A Survey on Simulators for Testing Self-Driving Cars
- Simulating Autonomous Driving in Massive Mixed Urban Traffic
- Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection
- Learning to simulate complex scenes
- WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving
- Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation
- CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
- Autonomics: In Search of a Foundation for Next Generation Autonomous Systems
- Reinforcement Learning with Latent Flow
- Unsupervised Metric Relocalization Using Transform Consistency Loss
- Digital Twins Are Not Monozygotic -- Cross-Replicating ADAS Testing in Two Industry-Grade Automotive Simulators
- Self-Supervised Steering Angle Prediction for Vehicle Control Using Visual Odometry
- Cooperative LIDAR Object Detection via Feature Sharing in Deep Networks
- Multiple criteria decision-making for lane-change model
- GLADAS: Gesture Learning for Advanced Driver Assistance Systems
- End-to-end Autonomous Driving Perception with Sequential Latent Representation Learning
- Dynamic Risk Assessment for Vehicles of Higher Automation Levels by Deep Learning
- Sensor Fusion: Gated Recurrent Fusion to Learn Driving Behavior from Temporal Multimodal Data
- Parallel and Multi-Objective Falsification with Scenic and VerifAI
- Addressing crash-imminent situations caused by human driven vehicle errors in a mixed traffic stream: a model-based reinforcement learning approach for CAV
- Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents
- Using a Supervised Method without supervision for foreground segmentation
- A gamified simulator and physical platform for self-driving algorithm training and validation
- Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360° Panoramic Imagery
- VUNet: Dynamic Scene View Synthesis for Traversability Estimation using an RGB Camera
- CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures
- Building Mental Models through Preview of Autopilot Behaviors
- Sidekick Policy Learning for Active Visual Exploration
- Monocular Direct Sparse Localization in a Prior 3D Surfel Map
- Self Training Autonomous Driving Agent
- Let's Keep It Safe: Designing User Interfaces that Allow Everyone to Contribute to AI Safety
- Multi-agent Interactive Prediction under Challenging Driving Scenarios
- Safer End-to-End Autonomous Driving via Conditional Imitation Learning and Command Augmentation
- 3D Surfel Map-Aided Visual Relocalization with Learned Descriptors
- Watch out for the risky actors: Assessing risk in dynamic environments for safe driving
- Risk-Aware Lane Selection on Highway with Dynamic Obstacles
- DASC: Towards A Road Damage-Aware Social-Media-Driven Car Sensing Framework for Disaster Response Applications
- Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory
- KIT Bus: A Shuttle Model for CARLA Simulator
- Convex Online Video Frame Subset Selection using Multiple Criteria for Data Efficient Autonomous Driving
- DriverGym: Democratising Reinforcement Learning for Autonomous Driving
- TrolleyMod v1.0: An Open-Source Simulation and Data-Collection Platform for Ethical Decision Making in Autonomous Vehicles
- Single-step Options for Adversary Driving
- Manual-Label Free 3D Detection via An Open-Source Simulator
- Video Class Agnostic Segmentation with Contrastive Learning for Autonomous Driving