Hybrid Skip: A Biologically Inspired Skip Connection for the UNet Architecture

Abstract

In this work we introduce a biologically inspired long-range skip connection for the UNet architecture that relies on the perceptual illusion of hybrid images, being images that simultaneously encode two images. The fusion of early encoder features with deeper decoder ones allows UNet models to produce finer-grained dense predictions. While proven in segmentation tasks, the network's benefits are down-weighted for dense regression tasks as these long-range skip connections additionally result in texture transfer artifacts. Specifically for depth estimation, this hurts smoothness and introduces false positive edges which are detrimental to the task due to the depth maps' piece-wise smooth nature. The proposed HybridSkip connections show improved performance in balancing the trade-off between edge preservation, and the minimization of texture transfer artifacts that hurt smoothness. This is achieved by the proper and balanced exchange of information that Hybrid-Skip connections offer between the high and low frequency, encoder and decoder features, respectively.

arXiv commentProject page at https://vcl3d.github.io/HybridSkip/

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Attention U-Net: Learning Where to Look for the Pancreas
• U${}^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection
• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
• Rigid-Motion Scattering for Texture Classification
• Modality specific U-Net variants for biomedical image segmentation: A survey
• W-Net: A Deep Model for Fully Unsupervised Image Segmentation
• Neural Light Transport for Relighting and View Synthesis
• Pano3D: A Holistic Benchmark and a Solid Baseline for $360^o$ Depth Estimation
• Distortion-aware Monocular Depth Estimation for Omnidirectional Images
• CNNs and Transformers Perceive Hybrid Images Similar to Humans
• KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation

Cited by in corpus (1)

• DwinFormer: Dual Window Transformers for End-to-End Monocular Depth Estimation