Hierarchical Neural Architecture Search for Deep Stereo Matching

Abstract

To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation. The underlying idea for the NAS algorithm is straightforward, namely, to enable the network the ability to choose among a set of operations (e.g., convolution with different filter sizes), one is able to find an optimal architecture that is better adapted to the problem at hand. However, so far the success of NAS has not been enjoyed by low-level geometric vision tasks such as stereo matching. This is partly due to the fact that state-of-the-art deep stereo matching networks, designed by humans, are already sheer in size. Directly applying the NAS to such massive structures is computationally prohibitive based on the currently available mainstream computing resources. In this paper, we propose the first end-to-end hierarchical NAS framework for deep stereo matching by incorporating task-specific human knowledge into the neural architecture search framework. Specifically, following the gold standard pipeline for deep stereo matching (i.e., feature extraction -- feature volume construction and dense matching), we optimize the architectures of the entire pipeline jointly. Extensive experiments show that our searched network outperforms all state-of-the-art deep stereo matching architectures and is ranked at the top 1 accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the top 1 on SceneFlow dataset with a substantial improvement on the size of the network and the speed of inference. The code is available at https://github.com/XuelianCheng/LEAStereo.

arXiv commentAccepted at NeurIPS 2020; Xuelian Cheng and Yiran Zhong made equal contribution

References in corpus (13)

• Neural Architecture Search with Reinforcement Learning
• EfficientDet: Scalable and Efficient Object Detection
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Self-Supervised Learning for Stereo Matching with Self-Improving Ability
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• Pyramid Stereo Matching Network
• GA-Net: Guided Aggregation Net for End-to-end Stereo Matching
• MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry
• Densely Connected Search Space for More Flexible Neural Architecture Search
• Customizable Architecture Search for Semantic Segmentation
• Open-World Stereo Video Matching with Deep RNN

Cited by in corpus (10)

• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Neural Markov Random Field for Stereo Matching
• DCVSMNet: Double Cost Volume Stereo Matching Network
• Towards Safer Robot-Assisted Surgery: A Markerless Augmented Reality Framework
• UAMD-Net: A Unified Adaptive Multimodal Neural Network for Dense Depth Completion
• Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching
• ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring
• LeanStereo: A Leaner Backbone based Stereo Network
• An evaluation of Deep Learning based stereo dense matching dataset shift from aerial images and a large scale stereo dataset
• Distilling Stereo Networks for Performant and Efficient Leaner Networks