Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark
arXiv:2104.12668
Abstract
Human gaze provides valuable information on human focus and intentions, making it a crucial area of research. Recently, deep learning has revolutionized appearance-based gaze estimation. However, due to the unique features of gaze estimation research, such as the unfair comparison between 2D gaze positions and 3D gaze vectors and the different pre-processing and post-processing methods, there is a lack of a definitive guideline for developing deep learning-based gaze estimation algorithms. In this paper, we present a systematic review of the appearance-based gaze estimation methods using deep learning. Firstly, we survey the existing gaze estimation algorithms along the typical gaze estimation pipeline: deep feature extraction, deep learning model design, personal calibration and platforms. Secondly, to fairly compare the performance of different approaches, we summarize the data pre-processing and post-processing methods, including face/eye detection, data rectification, 2D/3D gaze conversion and gaze origin conversion. Finally, we set up a comprehensive benchmark for deep learning-based gaze estimation. We characterize all the public datasets and provide the source code of typical gaze estimation algorithms. This paper serves not only as a reference to develop deep learning-based gaze estimation methods, but also a guideline for future gaze estimation research. The project web page can be found at https://phi-ai.buaa.edu.cn/Gazehub.
Accepted by TPAMI
References in corpus (13)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Overview of Multi-Task Learning in Deep Neural Networks
- It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation
- Deep Pictorial Gaze Estimation
- A Differential Approach for Gaze Estimation
- Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
- Eye Tracking for Everyone
- Towards Fast, Accurate and Stable 3D Dense Face Alignment
- Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues
- Domain Adaptation Gaze Estimation by Embedding with Prediction Consistency
- Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency
- Vulnerability of Appearance-based Gaze Estimation
- Learning to Detect Head Movement in Unconstrained Remote Gaze Estimation in the Wild