Visual-Inertial-Semantic Scene Representation for 3-D Object Detection
arXiv:1606.03968
Abstract
We describe a system to detect objects in three-dimensional space using video and inertial sensors (accelerometer and gyrometer), ubiquitous in modern mobile platforms from phones to drones. Inertials afford the ability to impose class-specific scale priors for objects, and provide a global orientation reference. A minimal sufficient representation, the posterior of semantic (identity) and syntactic (pose) attributes of objects in space, can be decomposed into a geometric term, which can be maintained by a localization-and-mapping filter, and a likelihood function, which can be approximated by a discriminatively-trained convolutional neural network. The resulting system can process the video stream causally in real time, and provides a representation of objects in the scene that is persistent: Confidence in the presence of objects grows with evidence, and objects previously seen are kept in memory even when temporarily occluded, with their return into view automatically predicted to prime re-detection.
To appear in CVPR 2017
References in corpus (9)
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
- Towards Scene Understanding with Detailed 3D Object Representations
- SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
- Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation
- Exploring Context with Deep Structured models for Semantic Segmentation
- On Invariance and Selectivity in Representation Learning
- Distributed Consistent Data Association