Correlated and Individual Multi-Modal Deep Learning for RGB-D Object Recognition
arXiv:1604.01655
Abstract
In this paper, we propose a new correlated and individual multi-modal deep learning (CIMDL) method for RGB-D object recognition. Unlike most conventional RGB-D object recognition methods which extract features from the RGB and depth channels individually, our CIMDL jointly learns feature representations from raw RGB-D data with a pair of deep neural networks, so that the sharable and modal-specific information can be simultaneously exploited. Specifically, we construct a pair of deep convolutional neural networks (CNNs) for the RGB and depth data, and concatenate them at the top layer of the network with a loss function which learns a new feature space where both correlated part and the individual part of the RGB-D information are well modelled. The parameters of the whole networks are updated by using the back-propagation criterion. Experimental results on two widely used RGB-D object image benchmark datasets clearly show that our method outperforms state-of-the-arts.
arXiv comment11 pages, 7 figures, submitted to a conference in 2016
References in corpus (5)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Deep Residual Learning for Image Recognition
- Indoor Semantic Segmentation using depth information
- Learning Multi-Domain Convolutional Neural Networks for Visual Tracking
- Deep Tracking: Visual Tracking Using Deep Convolutional Networks