Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

Abstract

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.

Paper submitted to Pattern Recognition Letters

References in corpus (3)

• Adam: A Method for Stochastic Optimization
• Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion
• Show, Attend and Interact: Perceivable Human-Robot Social Interaction through Neural Attention Q-Network

Cited by in corpus (9)

• Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
• Sample-Efficient Training of Robotic Guide Using Human Path Prediction Network
• Learning to Communicate Functional States with Nonverbal Expressions for Improved Human-Robot Collaboration
• Supporting User Autonomy with Multimodal Fusion to Detect when a User Needs Assistance from a Social Robot
• Probabilistic Radio-Visual Active Sensing for Search and Tracking
• Audio-Driven Reinforcement Learning for Head-Orientation in Naturalistic Environments
• Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Control
• Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View
• Learning Visual Voice Activity Detection with an Automatically Annotated Dataset