Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality

Abstract

Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.

arXiv comment16 pages, 10 figures, accepted to ETRA 2024 Full Papers

References in corpus (9)

• A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms
• PuRe: Robust pupil detection for real-time pervasive eye tracking
• Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments
• RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking
• PupilNet: Convolutional Neural Networks for Robust Pupil Detection
• EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking
• RIT-Eyes: Rendering of near-eye images for eye-tracking applications
• EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking
• 1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning