What Makes a Model Breathe? Understanding Reinforcement Learning Reward Function Design in Biomechanical User Simulation

Abstract

Biomechanical models allow for diverse simulations of user movements in interaction. Their performance depends critically on the careful design of reward functions, yet the interplay between reward components and emergent behaviours remains poorly understood. We investigate what makes a model "breathe" by systematically analysing the impact of rewarding effort minimisation, task completion, and target proximity on movement trajectories. Using a choice reaction task as a test-bed, we find that a combination of completion bonus and proximity incentives is essential for task success. Effort terms are optional, but can help avoid irregularities if scaled appropriately. Our work offers practical insights for HCI designers to create realistic simulations without needing deep reinforcement learning expertise, advancing the use of simulations as a powerful tool for interaction design and evaluation in HCI.

10 pages, 3 figures, 2 tables; CHI 25 LBW

References in corpus (5)

• Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity
• Optimal Feedback Control for Modeling Human-Computer Interaction
• SIM2VR: Towards Automated Biomechanical Testing in VR
• Reward Function Design for Crowd Simulation via Reinforcement Learning
• Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements

Cited by in corpus (4)

• Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCI
• Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCI
• Mind & Motion: Opportunities and Applications of Integrating Biomechanics and Cognitive Models in HCI
• MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning