Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping

Abstract

Assisting individuals in their daily activities through autonomous mobile robots, especially for users without specialized knowledge, is crucial. Specifically, the capability of robots to navigate to destinations based on human speech instructions is essential. While robots can take different paths to the same goal, the shortest path is not always the best. A preferred approach is to accommodate waypoint specifications flexibly, planning an improved alternative path, even with detours. Additionally, robots require real-time inference capabilities. This study aimed to realize a hierarchical spatial representation using a topometric semantic map and path planning with speech instructions, including waypoints. This paper presents Spatial Concept-based Topometric Semantic Mapping for Hierarchical Path Planning (SpCoTMHP), integrating place connectivity. This approach offers a novel integrated probabilistic generative model and fast approximate inference across hierarchy levels. A formulation based on control as probabilistic inference theoretically supports the proposed path planning algorithm. We conducted experiments in home environments using the Toyota Human Support Robot on the SIGVerse simulator and in a lab-office environment with the real robot, Albert. Users issued speech commands specifying the waypoint and goal, such as "Go to the bedroom via the corridor." Navigation experiments using speech instructions with a waypoint demonstrated a performance improvement of SpCoTMHP over the baseline hierarchical path planning method with heuristic path costs (HPP-I), in terms of the weighted success rate at which the robot reaches the closest target and passes the correct waypoints, by 0.590. The computation time was significantly accelerated by 7.14 seconds with SpCoTMHP compared to baseline HPP-I in advanced tasks.

arXiv commentAccepted to Frontiers in Robotics and AI

References in corpus (17)

• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• On Evaluation of Embodied Navigation Agents
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Symbol Emergence in Robotics: A Survey
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
• Symbol Emergence in Cognitive Developmental Systems: a Survey
• ChatGPT for Robotics: Design Principles and Model Abilities
• CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
• Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models
• Large Language Models for Robotics: A Survey
• Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping
• Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences
• Learning Geometrically-Constrained Hidden Markov Models for Robot Navigation: Bridging the Topological-Geometrical Gap
• Spatial Concept-Based Navigation with Human Speech Instructions via Probabilistic Inference on Bayesian Generative Model
• Topological Semantic Mapping by Consolidation of Deep Visual Features
• Active Exploration based on Information Gain by Particle Filter for Efficient Spatial Concept Formation