RoboBrain: Large-Scale Knowledge Engine for Robots

Abstract

In this paper we introduce a knowledge engine, which learns and shares knowledge representations, for robots to carry out a variety of tasks. Building such an engine brings with it the challenge of dealing with multiple data modalities including symbols, natural language, haptic senses, robot trajectories, visual features and many others. The \textit{knowledge} stored in the engine comes from multiple sources including physical interactions that robots have while performing tasks (perception, planning and control), knowledge bases from the Internet and learned representations from several robotics research groups. We discuss various technical aspects and associated challenges such as modeling the correctness of knowledge, inferring latent information and formulating different robotic tasks as queries to the knowledge engine. We describe the system architecture and how it supports different mechanisms for users and robots to interact with the engine. Finally, we demonstrate its use in three important research areas: grounding natural language, perception, and planning, which are the key building blocks for many robotic tasks. This knowledge engine is a collaborative effort and we call it RoboBrain.

10 pages, 9 figures

References in corpus (3)

• Contextually Guided Semantic Labeling and Search for 3D Point Clouds
• Learning Trajectory Preferences for Manipulators via Iterative Improvement
• Learning Object Arrangements in 3D Scenes using Human Context

Cited by in corpus (17)

• A Survey of Knowledge Representation in Service Robotics
• Functional Object-Oriented Network: Construction & Expansion
• Affordances in Robotic Tasks -- A Survey
• A Reference Software Architecture for Social Robots
• Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping
• Metarobotics for Industry and Society: Vision, Technologies, and Opportunities
• Reinforcement Learning for Semantic Segmentation in Indoor Scenes
• A Fog Robotics Approach to Deep Robot Learning: Application to Object Recognition and Grasp Planning in Surface Decluttering
• Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning
• Understanding User Instructions by Utilizing Open Knowledge for Service Robots
• Towards Robot-Centric Conceptual Knowledge Acquisition
• PlanIt: A Crowdsourcing Approach for Learning to Plan Paths from Large Scale Preference Feedback
• Relationship Oriented Affordance Learning through Manipulation Graph Construction
• Generalization to Novel Objects using Prior Relational Knowledge
• What Stands-in for a Missing Tool? A Prototypical Grounded Knowledge-based Approach to Tool Substitution
• Designing Interaction for Multi-agent Cooperative System in an Office Environment
• Leveraging Semantics for Incremental Learning in Multi-Relational Embeddings