Indoor Semantic Segmentation using depth information

Abstract

This work addresses multi-class segmentation of indoor scenes with RGB-D inputs. While this area of research has gained much attention recently, most works still rely on hand-crafted features. In contrast, we apply a multiscale convolutional network to learn features directly from the images and the depth information. We obtain state-of-the-art on the NYU-v2 depth dataset with an accuracy of 64.5%. We illustrate the labeling of indoor scenes in videos sequences that could be processed in real-time using appropriate hardware such as an FPGA.

8 pages, 3 figures

Cited by in corpus (40)

• Stacked Hourglass Networks for Human Pose Estimation
• Recent Advances in Convolutional Neural Networks
• RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation
• Graph Based Over-Segmentation Methods for 3D Point Clouds
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
• Deep Learning Based 3D Segmentation: A Survey
• 3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs
• Siamese Network for RGB-D Salient Object Detection and Beyond
• Semantic Pose using Deep Networks Trained on Synthetic RGB-D
• Multi-modal land cover mapping of remote sensing images using pyramid attention and gated fusion networks
• Discriminative Training of Deep Fully-connected Continuous CRF with Task-specific Loss
• 3D Gated Recurrent Fusion for Semantic Scene Completion
• Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection
• Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras
• Depth-aware CNN for RGB-D Segmentation
• Correlated and Individual Multi-Modal Deep Learning for RGB-D Object Recognition
• In Defense of the Direct Perception of Affordances
• Improving CNN-based Planar Object Detection with Geometric Prior Knowledge
• Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images
• Occlusion Edge Detection in RGB-D Frames using Deep Convolutional Networks
• Deep Tensor Convolution on Multicores
• Semantic Video Segmentation: A Review on Recent Approaches
• Multitask Learning with Single Gradient Step Update for Task Balancing
• LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement
• Auxiliary Learning by Implicit Differentiation
• RotoGrad: Gradient Homogenization in Multitask Learning
• Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks
• Normal Assisted Stereo Depth Estimation
• Multi${}^3$Net: Segmenting Flooded Buildings via Fusion of Multiresolution, Multisensor, and Multitemporal Satellite Imagery
• STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling
• Understand Scene Categories by Objects: A Semantic Regularized Scene Classifier Using Convolutional Neural Networks
• Deeply Exploit Depth Information for Object Detection
• GAPLE: Generalizable Approaching Policy LEarning for Robotic Object Searching in Indoor Environment
• Context Tricks for Cheap Semantic Segmentation
• Adversarial Structure Matching for Structured Prediction Tasks
• Scene Completeness-Aware Lidar Depth Completion for Driving Scenario
• Deep Action Sequence Learning for Causal Shape Transformation
• Deep feature selection-and-fusion for RGB-D semantic segmentation