Joint-ID: Transformer-based Joint Image Enhancement and Depth Estimation for Underwater Environments
arXiv:2611.22212 · doi:10.1109/JSEN.2023.3338730
Abstract
Underwater imaging is a challenging task due to factors such as scattering, absorption, and turbulence, which degrade image quality and limit visibility. In this paper, we propose a novel approach for enhancing underwater images that leverages the benefits of joint learning for simultaneous image enhancement and depth estimation. We introduce Joint-ID, a transformer-based neural network that can obtain high-perceptual image quality and depth information from raw underwater images. Our approach formulates a multi-modal objective function that addresses invalid depth, lack of sharpness, and image degradation based on color and local texture. We design an end-to-end training pipeline that enables joint restoration and depth estimation in a shared hierarchical feature space. Additionally, we propose a synthetic dataset with various distortions and scene depths for multi-task learning. We evaluate Joint-ID on synthetic and standard datasets, as well as real underwater images with diverse spectra and harsh turbidity, demonstrating its effectiveness for underwater image enhancement and depth estimation. Furthermore, we demonstrate the ability of Joint-ID to perform feature matching and saliency detection for visually guided underwater robots. Our proposed method has the potential to improve the visual perception of underwater environments and benefit applications such as oceanography and underwater robotics. Supplements are available at: https://sites.google.com/view/joint-id/home
arXiv comment11 pages. (c) 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
References in corpus (15)
- Decoupled Weight Decay Regularization
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Vision Transformers for Single Image Dehazing
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- U-shape Transformer for Underwater Image Enhancement
- From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation
- WaterGAN: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater Images
- DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation
- Deep Underwater Image Enhancement
- Laplacian-Steered Neural Style Transfer
- Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth
- All-In-One Underwater Image Enhancement using Domain-Adversarial Learning
- DIML/CVL RGB-D Dataset: 2M RGB-D Images of Natural Indoor and Outdoor Scenes
- DwinFormer: Dual Window Transformers for End-to-End Monocular Depth Estimation
- Single Underwater Image Enhancement Using an Analysis-Synthesis Network