Situationally-Aware Dynamics Learning

Abstract

Autonomous robots operating in complex, unstructured environments face significant challenges due to latent, unobserved factors that obscure their understanding of both their internal state and the external world. Addressing this challenge would enable robots to develop a more profound grasp of their operational context. To tackle this, we propose a novel framework for online learning of hidden state representations, with which the robots can adapt in real-time to uncertain and dynamic conditions that would otherwise be ambiguous and result in suboptimal or erroneous behaviors. Our approach is formalized as a Generalized Hidden Parameter Markov Decision Process, which explicitly models the influence of unobserved parameters on both transition dynamics and reward structures. Our core innovation lies in learning online the joint distribution of state transitions, which serves as an expressive representation of latent ego- and environmental-factors. This probabilistic approach supports the identification and adaptation to different operational situations, improving robustness and safety. Through a multivariate extension of Bayesian Online Changepoint Detection, our method segments changes in the underlying data generating process governing the robot's dynamics. The robot's transition model is then informed with a symbolic representation of the current situation derived from the joint distribution of latest state transitions, enabling adaptive and context-aware decision-making. To showcase the real-world effectiveness, we validate our approach in the challenging task of unstructured terrain navigation, where unmodeled and unmeasured terrain characteristics can significantly impact the robot's motion. Extensive experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies.

References in corpus (8)

• Learning Quadrupedal Locomotion over Challenging Terrain
• Robust Model Predictive Path Integral Control: Analysis and Performance Guarantees
• Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot
• CAHSOR: Competence-Aware High-Speed Off-Road Ground Navigation in SE(3)
• Traversability-aware Adaptive Optimization for Path Planning and Control in Mountainous Terrain
• Latent Representation in Human-Robot Interaction with Explicit Consideration of Periodic Dynamics
• Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains
• Fast Extrinsic Calibration for Multiple Inertial Measurement Units in Visual-Inertial System