Deep Image Homography Estimation

Abstract

We present a deep convolutional neural network for estimating the relative homography between a pair of images. Our feed-forward network has 10 layers, takes two stacked grayscale images as input, and produces an 8 degree of freedom homography which can be used to map the pixels from the first image to the second. We present two convolutional neural network architectures for HomographyNet: a regression network which directly estimates the real-valued homography parameters, and a classification network which produces a distribution over quantized homographies. We use a 4-point homography parameterization which maps the four corners from one image into the second image. Our networks are trained in an end-to-end fashion using warped MS-COCO images. Our approach works without the need for separate local feature detection and transformation estimation stages. Our deep models are compared to a traditional homography estimator based on ORB features and we highlight the scenarios where HomographyNet outperforms the traditional technique. We also describe a variety of applications powered by deep homography estimation, thus showcasing the flexibility of a deep learning approach.

RSS Workshop on Limits and Potentials of Deep Learning in Robotics

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Cited by in corpus (49)

• Deep 6-DOF Tracking
• UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor
• HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery
• Convolutional neural network architecture for geometric matching
• Wide-Baseline Relative Camera Pose Estimation with Directional Learning
• DeepVO: A Deep Learning approach for Monocular Visual Odometry
• Toward Geometric Deep SLAM
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• PFLD: A Practical Facial Landmark Detector
• Self-Improving Visual Odometry
• Visual Servoing from Deep Neural Networks
• Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography
• GCNv2: Efficient Correspondence Prediction for Real-Time SLAM
• Content-Aware Unsupervised Deep Homography Estimation
• Discriminative Optimization: Theory and Applications to Computer Vision Problems
• ContextDesc: Local Descriptor Augmentation with Cross-Modality Context
• Relative Camera Pose Estimation Using Convolutional Neural Networks
• Deep Cuboid Detection: Beyond 2D Bounding Boxes
• Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information
• DRMIME: Differentiable Mutual Information and Matrix Exponential for Multi-Resolution Image Registration
• RegNet: Learning the Optimization of Direct Image-to-Image Pose Registration
• DSAC - Differentiable RANSAC for Camera Localization
• Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction
• DeepMeshFlow: Content Adaptive Mesh Deformation for Robust Image Registration
• DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features
• Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentation
• Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation
• STN-Homography: estimate homography parameters directly
• Optimizing Through Learned Errors for Accurate Sports Field Registration
• Learning for Multi-Model and Multi-Type Fitting
• Learning Feature Descriptors using Camera Pose Supervision
• TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations
• Recovering Homography from Camera Captured Documents using Convolutional Neural Networks
• Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection
• Homography Estimation with Convolutional Neural Networks Under Conditions of Variance
• Following Gaze Across Views
• Deep Homography Estimation for Dynamic Scenes
• Deep Lucas-Kanade Homography for Multimodal Image Alignment
• An End to End Network Architecture for Fundamental Matrix Estimation
• Enhancing Multi-Robot Perception via Learned Data Association
• Learning Knowledge-Rich Sequential Model for Planar Homography Estimation in Aerial Video
• Deep Exposure Fusion with Deghosting via Homography Estimation and Attention Learning
• Driving among Flatmobiles: Bird-Eye-View occupancy grids from a monocular camera for holistic trajectory planning
• Adversarial Learning for Image Forensics Deep Matching with Atrous Convolution
• PARN: Pyramidal Affine Regression Networks for Dense Semantic Correspondence
• Learning for Multi-Type Subspace Clustering
• Learning-based Natural Geometric Matching with Homography Prior
• Real-time Halfway Domain Reconstruction of Motion and Geometry
• Homography augumented momentum constrastive learning for SAR image retrieval