The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
arXiv:1801.10202 · doi:10.1109/LRA.2018.2800793
Abstract
Event based cameras are a new passive sensing modality with a number of benefits over traditional cameras, including extremely low latency, asynchronous data acquisition, high dynamic range and very low power consumption. There has been a lot of recent interest and development in applying algorithms to use the events to perform a variety of 3D perception tasks, such as feature tracking, visual odometry, and stereo depth estimation. However, there currently lacks the wealth of labeled data that exists for traditional cameras to be used for both testing and development. In this paper, we present a large dataset with a synchronized stereo pair event based camera system, carried on a handheld rig, flown by a hexacopter, driven on top of a car and mounted on a motorcycle, in a variety of different illumination levels and environments. From each camera, we provide the event stream, grayscale images and IMU readings. In addition, we utilize a combination of IMU, a rigidly mounted lidar system, indoor and outdoor motion capture and GPS to provide accurate pose and depth images for each camera at up to 100Hz. For comparison, we also provide synchronized grayscale images and IMU readings from a frame based stereo camera system.
8 pages, 7 figures, 2 tables. Website: https://daniilidis-group.github.io/mvsec/. Video: https://www.youtube.com/watch?v=AwRMO5vFgak. Updated website and video in comments, DOI
References in corpus (2)
Cited by in corpus (78)
- Event-based Vision: A Survey
- EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
- Spiking Neural Networks Hardware Implementations and Challenges: a Survey
- Event-based Stereo Visual Odometry
- Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
- Semi-Dense 3D Reconstruction with a Stereo Event Camera
- Learning to Detect Objects with a 1 Megapixel Event Camera
- A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
- VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM
- Secrets of Event-Based Optical Flow
- ESVIO: Event-based Stereo Visual Inertial Odometry
- Learning Dense and Continuous Optical Flow from an Event Camera
- A Large Scale Event-based Detection Dataset for Automotive
- Event-based visual place recognition with ensembles of temporal windows
- Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera
- Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation
- Optical flow estimation from event-based cameras and spiking neural networks
- StereoSpike: Depth Learning with a Spiking Neural Network
- An Asynchronous Kalman Filter for Hybrid Event Cameras
- A Survey on RGB-D Datasets
- HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks
- Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
- A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera
- Event-based Background-Oriented Schlieren
- The GRIFFIN Perception Dataset: Bridging the Gap Between Flapping-Wing Flight and Robotic Perception
- Event Collapse in Contrast Maximization Frameworks
- ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
- Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
- Neuromorphic Perception and Navigation for Mobile Robots: A Review
- Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception
- Lightweight Event-based Optical Flow Estimation via Iterative Deblurring
- EVREAL: Towards a Comprehensive Benchmark and Analysis Suite for Event-based Video Reconstruction
- Fast Event-based Optical Flow Estimation by Triplet Matching
- Event-based Stereo Depth Estimation: A Survey
- EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance
- Reliable Object Tracking by Multimodal Hybrid Feature Extraction and Transformer-Based Fusion
- A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework
- Driving Datasets Literature Review
- Hardware/Software co-design with ADC-Less In-memory Computing Hardware for Spiking Neural Networks
- Event-based Stereo Visual Odometry with Native Temporal Resolution via Continuous-time Gaussian Process Regression
- EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision
- Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy
- hARMS: A Hardware Acceleration Architecture for Real-Time Event-Based Optical Flow
- Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
- Event Transformer+. A multi-purpose solution for efficient event data processing
- Evolutionary Spiking Neural Networks: A Survey
- Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data
- Simultaneous Localization and Mapping Related Datasets: A Comprehensive Survey
- DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System
- ES-PTAM: Event-based Stereo Parallel Tracking and Mapping
- SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
- Events-to-Video: Bringing Modern Computer Vision to Event Cameras
- Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
- Learning Parallax for Stereo Event-based Motion Deblurring
- v2e: From Video Frames to Realistic DVS Events
- EventNet: Asynchronous Recursive Event Processing
- On the Benefits of Visual Stabilization for Frame- and Event-based Perception
- Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion
- Neuromorphic Vision Data Coding: Classifying and Reviewing the Literature
- Asynchronous Event-Inertial Odometry using a Unified Gaussian Process Regression Framework
- Fast-Racing: An Open-source Strong Baseline for SE(3) Planning in Autonomous Drone Racing
- DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction
- Realtime Time Synchronized Event-based Stereo
- Event-VPR: End-to-End Weakly Supervised Network Architecture for Event-based Visual Place Recognition
- Hardware-Free Event Cameras Temporal Synchronization Based on Event Density Alignment
- Low-Latency Event-Based Velocimetry for Quadrotor Control in a Narrow Pipe
- FEDORA: Flying Event Dataset fOr Reactive behAvior
- Spatiotemporal Registration for Event-based Visual Odometry
- Comparing Representations in Tracking for Event Camera-based SLAM
- Single Image Optical Flow Estimation with an Event Camera
- Neuromorphic Visual Odometry System for Intelligent Vehicle Application with Bio-inspired Vision Sensor
- Event-Free Moving Object Segmentation from Moving Ego Vehicle
- Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction
- EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation
- Matching Neuromorphic Events and Color Images via Adversarial Learning
- OpenCL-based FPGA accelerator for disparity map generation with stereoscopic event cameras
- E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations
- Advanced Mapping Robot and High-Resolution Dataset