Instance Segmentation in the Dark

Abstract

Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. In this work, we take a deep look at instance segmentation in the dark and introduce several techniques that substantially boost the low-light inference accuracy. The proposed method is motivated by the observation that noise in low-light images introduces high-frequency disturbances to the feature maps of neural networks, thereby significantly degrading performance. To suppress this ``feature noise", we propose a novel learning method that relies on an adaptive weighted downsampling layer, a smooth-oriented convolutional block, and disturbance suppression learning. These components effectively reduce feature noise during downsampling and convolution operations, enabling the model to learn disturbance-invariant features. Furthermore, we discover that high-bit-depth RAW images can better preserve richer scene information in low-light conditions compared to typical camera sRGB outputs, thus supporting the use of RAW-input algorithms. Our analysis indicates that high bit-depth can be critical for low-light instance segmentation. To mitigate the scarcity of annotated RAW datasets, we leverage a low-light RAW synthetic pipeline to generate realistic low-light data. In addition, to facilitate further research in this direction, we capture a real-world low-light instance segmentation dataset comprising over two thousand paired low/normal-light images with instance-level pixel-wise annotations. Remarkably, without any image preprocessing, we achieve satisfactory performance on instance segmentation in very low light (4~\% AP higher than state-of-the-art competitors), meanwhile opening new opportunities for future research.

arXiv commentAccepted by International Journal of Computer Vision (IJCV) 2023

References in corpus (7)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Semantic Foggy Scene Understanding with Synthetic Data
• Deep Image Prior
• RENOIR - A Dataset for Real Low-Light Image Noise Reduction
• Night-time Scene Parsing with a Large Real Dataset
• Connecting Image Denoising and High-Level Vision Tasks via Deep Learning
• Hybrid supervised instance segmentation by learning label noise suppression

Cited by in corpus (6)

• A large-scale climate-aware satellite image dataset for domain adaptive land-cover semantic segmentation
• Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance
• Event-guided Low-light Video Semantic Segmentation
• A biologically inspired separable learning vision model for real-time traffic object perception in Dark
• ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation
• Multi-Scale Denoising in the Feature Space for Low-Light Instance Segmentation