Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle

Abstract

Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing. Despite significant advances, existing outside-the-vehicle referencing methods consider these modalities separately. Moreover, existing multimodal referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints. In this paper, we investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects (e.g. buildings) from the vehicle. We furthermore explore person-specific differences in this interaction by analyzing individuals' performance for pointing and gaze patterns, along with their effect on the driving task. Our statistical analysis shows significant differences in individual behaviour based on object's location (i.e. driver's right side vs. left side), object's surroundings, driving mode (i.e. autonomous vs. normal driving) as well as pointing and gaze duration, laying the foundation for a user-adaptive approach.

Cited by in corpus (12)

• A Design Space for Human Sensor and Actuator Focused In-Vehicle Interaction Based on a Systematic Literature Review
• AutoVis: Enabling Mixed-Immersive Analysis of Automotive User Interface Interaction Studies
• Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces
• What's on your mind? A Mental and Perceptual Load Estimation Framework towards Adaptive In-vehicle Interaction while Driving
• ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle
• It's all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight Camera
• Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers
• Multimodal Driver Referencing: A Comparison of Pointing to Objects Inside and Outside the Vehicle
• Augmented Journeys: Interactive Points of Interest for In-Car Augmented Reality
• Real-time Human Finger Pointing Recognition and Estimation for Robot Directives Using a Single Web-Camera
• Bumpy Ride? Understanding the Effects of External Forces on Spatial Interactions in Moving Vehicles
• Blending the Worlds: World-Fixed Visual Appearances in Automotive Augmented Reality