R${}^3$SGM: Real-time Raster-Respecting Semi-Global Matching for Power-Constrained Systems

Abstract

Stereo depth estimation is used for many computer vision applications. Though many popular methods strive solely for depth quality, for real-time mobile applications (e.g. prosthetic glasses or micro-UAVs), speed and power efficiency are equally, if not more, important. Many real-world systems rely on Semi-Global Matching (SGM) to achieve a good accuracy vs. speed balance, but power efficiency is hard to achieve with conventional hardware, making the use of embedded devices such as FPGAs attractive for low-power applications. However, the full SGM algorithm is ill-suited to deployment on FPGAs, and so most FPGA variants of it are partial, at the expense of accuracy. In a non-FPGA context, the accuracy of SGM has been improved by More Global Matching (MGM), which also helps tackle the streaking artifacts that afflict SGM. In this paper, we propose a novel, resource-efficient method that is inspired by MGM's techniques for improving depth quality, but which can be implemented to run in real time on a low-power FPGA. Through evaluation on multiple datasets (KITTI and Middlebury), we show that in comparison to other real-time capable stereo approaches, we can achieve a state-of-the-art balance between accuracy, power efficiency and speed, making our approach highly desirable for use in real-time systems with limited power.

arXiv commentAccepted in FPT 2018 as Oral presentation, 8 pages, 6 figures, 4 tables

References in corpus (7)

• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• Embedded real-time stereo estimation via Semi-Global Matching on the GPU
• InfiniTAM v3: A Framework for Large-Scale 3D Reconstruction with Loop Closure
• Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation
• CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
• Real-Time Dense Stereo Matching With ELAS on FPGA Accelerated Embedded Devices
• On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation

Cited by in corpus (2)

• ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices
• FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications