Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes

Abstract

Deep learning has enabled remarkable advances in scene understanding, particularly in semantic segmentation tasks. Yet, current state of the art approaches are limited to a closed set of classes, and fail when facing novel elements, also known as out of distribution (OoD) data. This is a problem as autonomous agents will inevitably come across a wide range of objects, all of which cannot be included during training. We propose a novel method to distinguish any object (foreground) from empty building structure (background) in indoor environments. We use normalizing flow to estimate the probability distribution of high-dimensional background descriptors. Foreground objects are therefore detected as areas in an image for which the descriptors are unlikely given the background distribution. As our method does not explicitly learn the representation of individual objects, its performance generalizes well outside of the training examples. Our model results in an innovative solution to reliably segment foreground from background in indoor scenes, which opens the way to a safer deployment of robots in human environments.

arXiv commentPreprint version after acceptance of publication in the IEEE robotics and automation letters

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Fully Convolutional Networks for Semantic Segmentation
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• NICE: Non-linear Independent Components Estimation
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Distance-based Confidence Score for Neural Network Classifiers
• Do Deep Generative Models Know What They Don't Know?
• Fusion++: Volumetric Object-Level SLAM