Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCI
arXiv:2508.15727 · doi:10.1145/3746059.3747779
Abstract
Designing effective reward functions is critical for reinforcement learning-based biomechanical simulations, yet HCI researchers and practitioners often waste (computation) time with unintuitive trial-and-error tuning. This paper demystifies reward function design by systematically analyzing the impact of effort minimization, task completion bonuses, and target proximity incentives on typical HCI tasks such as pointing, tracking, and choice reaction. We show that proximity incentives are essential for guiding movement, while completion bonuses ensure task success. Effort terms, though optional, help refine motion regularity when appropriately scaled. We perform an extensive analysis of how sensitive task success and completion time depend on the weights of these three reward components. From these results we derive practical guidelines to create plausible biomechanical simulations without the need for reinforcement learning expertise, which we then validate on remote control and keyboard typing tasks. This paper advances simulation-based interaction design and evaluation in HCI by improving the efficiency and applicability of biomechanical user modeling for real-world interface development.
17 pages, 14 figures, 1 table, ACM UIST 2025
References in corpus (10)
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
- Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
- Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity
- Optimal Feedback Control for Modeling Human-Computer Interaction
- SIM2VR: Towards Automated Biomechanical Testing in VR
- Empirical Design in Reinforcement Learning
- Reward Function Design for Crowd Simulation via Reinforcement Learning
- Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements
- What Makes a Model Breathe? Understanding Reinforcement Learning Reward Function Design in Biomechanical User Simulation
- Towards Improving Reward Design in RL: A Reward Alignment Metric for RL Practitioners