Are we done with object recognition? The iCub robot's perspective

Abstract

We report on an extensive study of the benefits and limitations of current deep learning approaches to object recognition in robot vision scenarios, introducing a novel dataset used for our investigation. To avoid the biases in currently available datasets, we consider a natural human-robot interaction setting to design a data-acquisition protocol for visual object recognition on the iCub humanoid robot. Analyzing the performance of off-the-shelf models trained off-line on large-scale image retrieval datasets, we show the necessity for knowledge transfer. We evaluate different ways in which this last step can be done, and identify the major bottlenecks affecting robotic scenarios. By studying both object categorization and identification problems, we highlight key differences between object recognition in robotics applications and in image retrieval tasks, for which the considered deep learning approaches have been originally designed. In a nutshell, our results confirm the remarkable improvements yield by deep learning in this setting, while pointing to specific open challenges that need be addressed for seamless deployment in robotics.

21 pages + supplementary material

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Improving neural networks by preventing co-adaptation of feature detectors
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• ShapeNet: An Information-Rich 3D Model Repository
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• Understanding Neural Networks Through Deep Visualization
• End-to-End Training of Deep Visuomotor Policies
• Generalization Properties of Learning with Random Features
• Scene recognition with CNNs: objects, scales and dataset bias
• Deep Image Retrieval: Learning global representations for image search
• One-Shot Adaptation of Supervised Deep Convolutional Models
• Learning to Linearize Under Uncertainty
• Towards Adapting ImageNet to Reality: Scalable Domain Adaptation with Implicit Low-rank Transformations
• Improved Deep Learning of Object Category using Pose Information
• Learning to Recognize Objects by Retaining other Factors of Variation

Cited by in corpus (6)

• Uncertainty for Identifying Open-Set Errors in Visual Object Detection
• A wearable sensor vest for social humanoid robots with GPGPU, IoT, and modular software architecture
• Weakly-Supervised Object Detection Learning through Human-Robot Interaction
• Fast Region Proposal Learning for Object Detection for Robotics
• Fast Object Segmentation Learning with Kernel-based Methods for Robotics
• What's in the Black Box? The False Negative Mechanisms Inside Object Detectors