STATIC: Surface-normal-guided Two-branch Alignment and Temporal Integration for Consistent Video Depth Estimation

Abstract

Video depth estimation is essential for applications such as autonomous driving, AR/VR, and robotics. Recent transformer-based single-image monocular depth estimation models perform well on a single image but struggle with depth consistency across video frames. Traditional methods aim to improve temporal consistency using multi-frame temporal modules or explicit priors like optical flow and camera parameters. However, these approaches face issues such as high memory use, reduced performance with dynamic or irregular motion, and a heavy reliance on the accuracy of external motion estimators. We propose STATIC, a novel model that independently learns temporal consistency in geometrically stable and unstable regions using only two adjacent frames. By leveraging intrinsic geometric cues rather than external motion priors, STATIC generates a depth-derived geometry-aware difference mask from surface normals to identify regions with inter-frame geometric variation by measuring directional variance. For geometrically stable regions, the Masked Stable (MS) module enhances temporal consistency by focusing on structurally stable regions. For geometrically unstable regions, the Surface Normal Similarity (SNS) module models inter-frame correspondences and ensures temporal consistency by measuring feature similarities between frames. A final refinement integrates the independently learned stable and unstable features, enabling STATIC to improve consistent depth prediction across consecutive frames. Experiments on KITTI, NYUv2, and Sintel demonstrate strong depth accuracy and temporal consistency, as well as stable performance across the tested frame-rate and camera-motion conditions.

arXiv commentAccepted by Pattern Recognition