Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity

Abstract

Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts' Law and the 2/3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.

19 pages, 7 figures

References in corpus (6)

• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
• Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation
• Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity
• Converting Biomechanical Models from OpenSim to MuJoCo
• Reinforcement learning with world model

Cited by in corpus (9)

• Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity
• Optimal Feedback Control for Modeling Human-Computer Interaction
• SIM2VR: Towards Automated Biomechanical Testing in VR
• Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCI
• Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements
• What Makes a Model Breathe? Understanding Reinforcement Learning Reward Function Design in Biomechanical User Simulation
• Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCI
• MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning
• Mind & Motion: Opportunities and Applications of Integrating Biomechanics and Cognitive Models in HCI