TartanAir: A Dataset to Push the Limits of Visual SLAM

Abstract

We present a challenging dataset, the TartanAir, for robot navigation tasks and more. The data is collected in photo-realistic simulation environments with the presence of moving objects, changing light and various weather conditions. By collecting data in simulations, we are able to obtain multi-modal sensor data and precise ground truth labels such as the stereo RGB image, depth image, segmentation, optical flow, camera poses, and LiDAR point cloud. We set up large numbers of environments with various styles and scenes, covering challenging viewpoints and diverse motion patterns that are difficult to achieve by using physical data collection platforms. In order to enable data collection at such a large scale, we develop an automatic pipeline, including mapping, trajectory sampling, data processing, and data verification. We evaluate the impact of various factors on visual SLAM algorithms using our data. The results of state-of-the-art algorithms reveal that the visual SLAM problem is far from solved. Methods that show good performance on established datasets such as KITTI do not perform well in more difficult scenarios. Although we use the simulation, our goal is to push the limits of Visual SLAM algorithms in the real world by providing a challenging benchmark for testing new methods, while also using a large diverse training data for learning-based methods. Our dataset is available at \url{http://theairlab.org/tartanair-dataset}.

arXiv commentAccepted by IROS 2020

References in corpus (4)

• The TUM VI Benchmark for Evaluating Visual-Inertial Odometry
• Deep Drone Racing: From Simulation to Reality with Domain Randomization
• Keyframe-based monocular SLAM: design, survey, and future directions
• Non-Iterative SLAM

Cited by in corpus (10)

• DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
• EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association
• RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching
• Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference
• IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation
• Simultaneous Localization and Mapping Related Datasets: A Comprehensive Survey
• VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enough
• Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry
• Tangent Space Backpropagation for 3D Transformation Groups
• Asynchronous Multi-View SLAM