Visuo-Haptic Object Perception for Robots: An Overview

Abstract

The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.

published in Autonomous Robots

References in corpus (5)

• DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
• A Survey of Robot Manipulation in Contact
• A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration
• Finding safe 3D robot grasps through efficient haptic exploration with unscented Bayesian optimization and collision penalty
• A Biomimetic Fingerprint for Robotic Tactile Sensing

Cited by in corpus (5)

• Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies
• Do Robots Really Need Anthropomorphic Hands? A Comparison of Human and Robotic Hands
• ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception
• Optimizing BioTac Simulation for Realistic Tactile Perception
• SPLIT: Separating Physical-Contact via Latent Arithmetic in Image-Based Tactile Sensors