A Comprehensive Review of Modern Object Segmentation Approaches

Abstract

Image segmentation is the task of associating pixels in an image with their respective object class labels. It has a wide range of applications in many industries including healthcare, transportation, robotics, fashion, home improvement, and tourism. Many deep learning-based approaches have been developed for image-level object recognition and pixel-level scene understanding-with the latter requiring a much denser annotation of scenes with a large set of objects. Extensions of image segmentation tasks include 3D and video segmentation, where units of voxels, point clouds, and video frames are classified into different objects. We use "Object Segmentation" to refer to the union of these segmentation tasks. In this monograph, we investigate both traditional and modern object segmentation approaches, comparing their strengths, weaknesses, and utilities. We examine in detail the wide range of deep learning-based segmentation techniques developed in recent years, provide a review of the widely used datasets and evaluation metrics, and discuss potential future research directions.

arXiv comment173 pages, 49 figures, published in Foundations and Trends in Computer Graphics and Vision on 10/4/22. Authors retain copyright

References in corpus (57)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Neural Architecture Search with Reinforcement Learning
• Fully Convolutional Networks for Semantic Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• ShapeNet: An Information-Rich 3D Model Repository
• Squeeze-and-Excitation Networks
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Neural Architecture Search: A Survey
• Semantic Foggy Scene Understanding with Synthetic Data
• ParseNet: Looking Wider to See Better
• What makes for effective detection proposals?
• Learning to Segment Object Candidates
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• The ApolloScape Open Dataset for Autonomous Driving and its Application
• YOLACT++: Better Real-time Instance Segmentation
• Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
• Semantic Segmentation using Adversarial Networks
• SOLOv2: Dynamic and Fast Instance Segmentation
• Rethinking Pre-training and Self-training
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Hierarchical Multi-Scale Attention for Semantic Segmentation
• Deep Human Parsing with Active Template Regression
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics
• YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
• A2D2: Audi Autonomous Driving Dataset
• Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• Semantic Instance Segmentation via Deep Metric Learning
• K-Net: Towards Unified Image Segmentation
• DeeperLab: Single-Shot Image Parser
• Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras
• End-to-End Object Detection with Adaptive Clustering Transformer
• Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models
• Light-Weight RefineNet for Real-Time Semantic Segmentation
• Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusion
• H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
• HoME: a Household Multimodal Environment
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• SOLQ: Segmenting Objects by Learning Queries
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
• Global Aggregation then Local Distribution in Fully Convolutional Networks
• Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation
• Scaling Wide Residual Networks for Panoptic Segmentation
• PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers
• DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
• Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime
• A Survey on Deep Learning Technique for Video Segmentation
• Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
• Rethinking Dilated Convolution for Real-time Semantic Segmentation
• PanoNet: Real-time Panoptic Segmentation through Position-Sensitive Feature Embedding
• Tracking Instances as Queries
• Unifying Instance and Panoptic Segmentation with Dynamic Rank-1 Convolutions
• Bipartite Conditional Random Fields for Panoptic Segmentation

Cited by in corpus (1)

• FusionVision: A comprehensive approach of 3D object reconstruction and segmentation from RGB-D cameras using YOLO and fast segment anything