Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies

Abstract

Humans leverage multiple sensor modalities when interacting with objects and discovering their intrinsic properties. Using the visual modality alone is insufficient for deriving intuition behind object properties (e.g., which of two boxes is heavier), making it essential to consider non-visual modalities as well, such as the tactile and auditory. Whereas robots may leverage various modalities to obtain object property understanding via learned exploratory interactions with objects (e.g., grasping, lifting, and shaking behaviors), challenges remain: the implicit knowledge acquired by one robot via object exploration cannot be directly leveraged by another robot with different morphology, because the sensor models, observed data distributions, and interaction capabilities are different across these different robot configurations. To avoid the costly process of learning interactive object perception tasks from scratch, we propose a multi-stage projection framework for each new robot for transferring implicit knowledge of object properties across heterogeneous robot morphologies. We evaluate our approach on the object-property recognition and object-identity recognition tasks, using a dataset containing two heterogeneous robots that perform 7,600 object interactions. Results indicate that knowledge can be transferred across robots, such that a newly-deployed robot can bootstrap its recognition models without exhaustively exploring all objects. We also propose a data augmentation technique and show that this technique improves the generalization of models. We release our code and datasets, here: https://github.com/gtatiya/Implicit-Knowledge-Transfer.

arXiv commentIn proceedings of the IEEE International Conference on Robotics and Automation (ICRA), May 29 - June 2, 2023 , ExCeL London, UK

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Interactive Perception: Leveraging Action in Perception and Perception in Action
• Kernel Manifold Alignment
• Visuo-Haptic Object Perception for Robots: An Overview
• Bayesian and Neural Inference on LSTM-based Object Recognition from Tactile and Kinesthetic Information
• A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration

Cited by in corpus (1)

• Single-grasp deformable object discrimination: the effect of gripper morphology, sensing modalities, and action parameters