Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
arXiv:1607.03468 · doi:10.1109/TPAMI.2017.2769655
Abstract
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that---because of the technological advantages of the event camera---our pipeline works in scenes characterized by high-speed motion, which are still unaccessible to standard cameras.
12 pages, 13 figures. 2 tables. (in press)
References in corpus (2)
Cited by in corpus (29)
- Event-based Vision: A Survey
- Event-based Stereo Visual Odometry
- Continuous-Time Visual-Inertial Odometry for Event Cameras
- Asynchronous, Photometric Feature Tracking using Events and Frames
- Event-Based Motion Segmentation by Motion Compensation
- Focus Is All You Need: Loss Functions For Event-based Vision
- Event-aided Direct Sparse Odometry
- Event-based Motion Segmentation with Spatio-Temporal Graph Cuts
- Learning Dense and Continuous Optical Flow from an Event Camera
- End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking
- Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking
- E-MLB: Multilevel Benchmark for Event-Based Camera Denoising
- CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
- ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
- Neuromorphic Perception and Navigation for Mobile Robots: A Review
- On Quasi-Isometry of Threshold-Based Sampling
- EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision
- ABMOF: A Novel Optical Flow Algorithm for Dynamic Vision Sensors
- Data-Driven Feature Tracking for Event Cameras With and Without Frames
- Image Based Camera Localization: an Overview
- An N-Point Linear Solver for Line and Motion Estimation with Event Cameras
- Events-to-Video: Bringing Modern Computer Vision to Event Cameras
- Continuous Event-Line Constraint for Closed-Form Velocity Initialization
- EventHPE: Event-based 3D Human Pose and Shape Estimation
- Globally Optimal Contrast Maximisation for Event-based Motion Estimation
- Comparing Representations in Tracking for Event Camera-based SLAM
- Event Data Association via Robust Model Fitting for Event-based Object Tracking
- ESL: Event-based Structured Light
- Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction