RIT-Eyes: Rendering of near-eye images for eye-tracking applications

Abstract

Deep neural networks for video-based eye tracking have demonstrated resilience to noisy environments, stray reflections, and low resolution. However, to train these networks, a large number of manually annotated images are required. To alleviate the cumbersome process of manual labeling, computer graphics rendering is employed to automatically generate a large corpus of annotated eye images under various conditions. In this work, we introduce a synthetic eye image generation platform that improves upon previous work by adding features such as an active deformable iris, an aspherical cornea, retinal retro-reflection, gaze-coordinated eye-lid deformations, and blinks. To demonstrate the utility of our platform, we render images reflecting the represented gaze distributions inherent in two publicly available datasets, NVGaze and OpenEDS. We also report on the performance of two semantic segmentation architectures (SegNet and RITnet) trained on rendered images and tested on the original datasets.

References in corpus (6)

• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• Appearance-Based Gaze Estimation in the Wild
• Deep Pictorial Gaze Estimation
• Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings
• Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities
• RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking

Cited by in corpus (4)

• EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking
• Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)
• LEyes: A Lightweight Framework for Deep Learning-Based Eye Tracking using Synthetic Eye Images
• Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality