The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM

Abstract

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array. These sensors have great potential for high-speed robotics and computer vision because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and very high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. For this purpose, we present and release a collection of datasets captured with a DAVIS in a variety of synthetic and real environments, which we hope will motivate research on new algorithms for high-speed and high-dynamic-range robotics and computer-vision applications. In addition to global-shutter intensity images and asynchronous events, we provide inertial measurements and ground-truth camera poses from a motion-capture system. The latter allows comparing the pose accuracy of ego-motion estimation algorithms quantitatively. All the data are released both as standard text files and binary files (i.e., rosbag). This paper provides an overview of the available data and describes a simulator that we release open-source to create synthetic event-camera data.

arXiv comment7 pages, 4 figures, 3 tables

Cited by in corpus (68)

• Event-based Vision: A Survey
• The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
• Event-based Stereo Visual Odometry
• Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
• Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception
• Continuous-Time Visual-Inertial Odometry for Event Cameras
• Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks
• Asynchronous, Photometric Feature Tracking using Events and Frames
• Event-Based Motion Segmentation by Motion Compensation
• Graph-based Spatial-temporal Feature Learning for Neuromorphic Vision Sensing
• Focus Is All You Need: Loss Functions For Event-based Vision
• Semi-Dense 3D Reconstruction with a Stereo Event Camera
• Visual SLAM: What are the Current Trends and What to Expect?
• The Hilti SLAM Challenge Dataset
• VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM
• Secrets of Event-Based Optical Flow
• Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision
• Asynchronous Spatial Image Convolutions for Event Cameras
• Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking
• CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
• An Asynchronous Kalman Filter for Hybrid Event Cameras
• Inceptive Event Time-Surfaces for Object Classification Using Neuromorphic Cameras
• Globally-Optimal Contrast Maximisation for Event Cameras
• HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks
• IDOL: A Framework for IMU-DVS Odometry using Lines
• Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
• LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor
• The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data
• The GRIFFIN Perception Dataset: Bridging the Gap Between Flapping-Wing Flight and Robotic Perception
• Event Collapse in Contrast Maximization Frameworks
• Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow
• Hough2Map -- Iterative Event-based Hough Transform for High-Speed Railway Mapping
• Neuromorphic Imaging with Joint Image Deblurring and Event Denoising
• Photorealistic Image Reconstruction from Hybrid Intensity and Event based Sensor
• Fast Event-based Optical Flow Estimation by Triplet Matching
• EVREAL: Towards a Comprehensive Benchmark and Analysis Suite for Event-based Video Reconstruction
• Asynchronous Blob Tracker for Event Cameras
• Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environment
• A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework
• EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance
• hARMS: A Hardware Acceleration Architecture for Real-Time Event-Based Optical Flow
• EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision
• RN-Net: Reservoir Nodes-Enabled Neuromorphic Vision Sensing Network
• Evolution of Visual Odometry Techniques
• An Asynchronous Linear Filter Architecture for Hybrid Event-Frame Cameras
• A Quantum Leaky Integrate-and-Fire Spiking Neuron and Network
• Data-Driven Feature Tracking for Event Cameras With and Without Frames
• Visual Odometry with Neuromorphic Resonator Networks
• Neuromorphic Imaging with Super-Resolution
• Benchmarking Conventional Vision Models on Neuromorphic Fall Detection and Action Recognition Dataset
• Event Encryption: Rethinking Privacy Exposure for Neuromorphic Imaging
• An Asynchronous Intensity Representation for Framed and Event Video Sources
• On the Benefits of Visual Stabilization for Frame- and Event-based Perception
• Asynchronous Event-Inertial Odometry using a Unified Gaussian Process Regression Framework
• Using The Feedback of Dynamic Active-Pixel Vision Sensor (Davis) to Prevent Slip in Real Time
• Hardware-Free Event Cameras Temporal Synchronization Based on Event Density Alignment
• Event-based Photometric Bundle Adjustment
• Event-based Mosaicing Bundle Adjustment
• Event-based Visual Tracking in Dynamic Environments
• Spatiotemporal Flux Probing for Single-Photon Videography
• AsynEIO: Asynchronous Monocular Event-Inertial Odometry Using Gaussian Process Regression
• FPGA-Based Hardware Architecture for Contrast Maximization in Event-Based Vision
• E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations
• LuvHarris: A Practical Corner Detector for Event-cameras