Simultaneous multi-view instance detection with learned geometric soft-constraints

Abstract

We propose to jointly learn multi-view geometry and warping between views of the same object instances for robust cross-view object detection. What makes multi-view object instance detection difficult are strong changes in viewpoint, lighting conditions, high similarity of neighbouring objects, and strong variability in scale. By turning object detection and instance re-identification in different views into a joint learning task, we are able to incorporate both image appearance and geometric soft constraints into a single, multi-view detection process that is learnable end-to-end. We validate our method on a new, large data set of street-level panoramas of urban objects and show superior performance compared to various baselines. Our contribution is threefold: a large-scale, publicly available data set for multi-view instance detection and re-identification; an annotation tool custom-tailored for multi-view instance detection; and a novel, holistic multi-view instance detection and re-identification method that jointly models geometry and appearance across views.

arXiv commentInternationcal Conference on Computer Vision 2019 (ICCV 19)

References in corpus (5)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
• From Google Maps to a Fine-Grained Catalog of Street trees
• Towards seamless multi-view scene analysis from satellite to street-level

Cited by in corpus (5)

• DOLPHINS: Dataset for Collaborative Perception enabled Harmonious and Interconnected Self-driving
• End-to-end Learning Improves Static Object Geo-localization in Monocular Video
• STRIDE: Street View-based Environmental Feature Detection and Pedestrian Collision Prediction
• Descriptor-Free Multi-View Region Matching for Instance-Wise 3D Reconstruction
• GeoGraph: Learning graph-based multi-view object detection with geometric cues end-to-end