What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?

Abstract

The finding that very large networks can be trained efficiently and reliably has led to a paradigm shift in computer vision from engineered solutions to learning formulations. As a result, the research challenge shifts from devising algorithms to creating suitable and abundant training data for supervised learning. How to efficiently create such training data? The dominant data acquisition method in visual recognition is based on web data and manual annotation. Yet, for many computer vision problems, such as stereo or optical flow estimation, this approach is not feasible because humans cannot manually enter a pixel-accurate flow field. In this paper, we promote the use of synthetically generated data for the purpose of training deep networks on such tasks.We suggest multiple ways to generate such data and evaluate the influence of dataset properties on the performance and generalization properties of the resulting networks. We also demonstrate the benefit of learning schedules that use different types of data at selected stages of the training process.

arXiv commentadded references (UCL dataset); added IJCV copyright information

References in corpus (2)

• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Cited by in corpus (27)

• A Survey on Deep Learning Techniques for Stereo-based Depth Estimation
• How much real data do we actually need: Analyzing object detection performance using synthetic and real data
• Ethical behavior in humans and machines -- Evaluating training data quality for beneficial machine learning
• WHU-Stereo: A Challenging Benchmark for Stereo Matching of High-Resolution Satellite Images
• MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation
• TrimBot2020: an outdoor robot for automatic gardening
• How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study
• RarePlanes: Synthetic Data Takes Flight
• Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenes
• Towards Better Generalization: Joint Depth-Pose Learning without PoseNet
• BIM Hyperreality: Data Synthesis Using BIM and Hyperrealistic Rendering for Deep Learning
• AutoFlow: Learning a Better Training Set for Optical Flow
• Leveraging synthetic imagery for collision-at-sea avoidance
• Automatic Generation of Dense Non-rigid Optical Flow
• Continual Occlusions and Optical Flow Estimation
• Learning to Generate Synthetic 3D Training Data through Hybrid Gradient
• A Deep Temporal Fusion Framework for Scene Flow Using a Learnable Motion Model and Occlusions
• LEAD: LiDAR Extender for Autonomous Driving
• DynaDog+T: A Parametric Animal Model for Synthetic Canine Image Generation
• Improving Depth Estimation using Location Information
• Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image
• Region Deformer Networks for Unsupervised Depth Estimation from Unconstrained Monocular Videos
• Semi-synthesis: A fast way to produce effective datasets for stereo matching
• Learned Dual-View Reflection Removal
• Learning to See by Looking at Noise
• On the Synergies between Machine Learning and Binocular Stereo for Depth Estimation from Images: a Survey
• On the Real-World Generalisability of Optical Flow Models