GEM: Group Enhanced Model for Learning Dynamical Control Systems

Abstract

Learning the dynamics of a physical system wherein an autonomous agent operates is an important task. Often these systems present apparent geometric structures. For instance, the trajectories of a robotic manipulator can be broken down into a collection of its transitional and rotational motions, fully characterized by the corresponding Lie groups and Lie algebras. In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning. We hypothesize that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. To verify this hypothesis, we introduce the Group Enhanced Model (GEM). GEMs significantly outperform conventional transition models on tasks of long-term prediction, planning, and model-based reinforcement learning across a diverse suite of standard continuous-control environments, including Walker, Hopper, Reacher, Half-Cheetah, Inverted Pendulums, Ant, and Humanoid. Furthermore, plugging GEM into existing state of the art systems enhances their performance, which we demonstrate on the PETS system. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions and practical applications along this direction. Our code is publicly available at: https://tinyurl.com/GEMMBRL.

arXiv comment14 pages, 8 figures

References in corpus (12)

• Neural Ordinary Differential Equations
• Group Equivariant Convolutional Networks
• DeepMind Control Suite
• Hamiltonian Neural Networks
• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• When to Trust Your Model: Model-Based Policy Optimization
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• A Unified Bellman Equation for Causal Information and Value in Markov Decision Processes
• B-Spline CNNs on Lie Groups
• Learning Group Structure and Disentangled Representations of Dynamical Environments
• Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning