Joint Implicit Image Function for Guided Depth Super-Resolution

Abstract

Guided depth super-resolution is a practical task where a low-resolution and noisy input depth map is restored to a high-resolution version, with the help of a high-resolution RGB guide image. Existing methods usually view this task as a generalized guided filtering problem that relies on designing explicit filters and objective functions, or a dense regression problem that directly predicts the target image via deep neural networks. These methods suffer from either model capability or interpretability. Inspired by the recent progress in implicit neural representation, we propose to formulate the guided super-resolution as a neural implicit image interpolation problem, where we take the form of a general image interpolation but use a novel Joint Implicit Image Function (JIIF) representation to learn both the interpolation weights and values. JIIF represents the target image domain with spatially distributed local latent codes extracted from the input image and the guide image, and uses a graph attention mechanism to learn the interpolation weights at the same time in one unified deep implicit function. We demonstrate the effectiveness of our JIIF representation on guided depth super-resolution task, significantly outperforming state-of-the-art methods on three public benchmarks. Code can be found at \url{https://git.io/JC2sU}.

arXiv commentAccepted by ACM MM 2021

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Residual Dense Network for Image Super-Resolution
• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
• Deformable Kernel Networks for Joint Image Filtering
• Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
• Pixel-Adaptive Convolutional Neural Networks
• 3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
• A Deep Primal-Dual Network for Guided Depth Super-Resolution
• A Joint Intensity and Depth Co-Sparse Analysis Model for Depth Map Super-Resolution

Cited by in corpus (5)

• An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation
• Self-Supervised Coordinate Projection Network for Sparse-View Computed Tomography
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution
• Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors
• IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution