Interactive Perception: Leveraging Action in Perception and Perception in Action

Abstract

Recent approaches in robotics follow the insight that perception is facilitated by interaction with the environment. These approaches are subsumed under the term of Interactive Perception (IP). It provides the following benefits: (i) interaction with the environment creates a rich sensory signal that would otherwise not be present and (ii) knowledge of the regularity in the combined space of sensory data and action parameters facilitate the prediction and interpretation of the signal. In this survey we postulate this as a principle and collect evidence in support by analyzing and categorizing existing work in this area. We also provide an overview of the most important applications of Interactive Perception. We close this survey by discussing remaining open questions. Thereby, we hope to define a field and inspire future work.

Equal contribution by first three authors

References in corpus (1)

• A Probabilistic Framework for Learning Kinematic Models of Articulated Objects

Cited by in corpus (16)

• State Representation Learning for Control: An Overview
• Proximity Perception in Human-Centered Robotics: A Survey on Sensing Systems and Applications
• A Survey of Robot Manipulation in Contact
• Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter
• GapFlyt: Active Vision Based Minimalist Structure-less Gap Detection For Quadrotor Flight
• Active Visuo-Tactile Interactive Robotic Perception for Accurate Object Pose Estimation in Dense Clutter
• From Screens to Scenes: A Survey of Embodied AI in Healthcare
• Motion-based Object Segmentation based on Dense RGB-D Scene Flow
• MQA: Answering the Question via Robotic Manipulation
• Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies
• Object and Relation Centric Representations for Push Effect Prediction
• Inferring 3D Shapes of Unknown Rigid Objects in Clutter through Inverse Physics Reasoning
• A Robotics-Inspired Scanpath Model Reveals the Importance of Uncertainty and Semantic Object Cues for Gaze Guidance in Dynamic Scenes
• ROSGPT_Vision: Commanding Robots Using Only Language Models' Prompts
• No Plan but Everything Under Control: Robustly Solving Sequential Tasks with Dynamically Composed Gradient Descent
• CAT-RRT: Motion Planning that Admits Contact One Link at a Time