765 citations · 1.7k across the 25 of their papers we have counts for
32 papers
ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video
Xiaozhong Lyu, Gen Li, Zhiyin Qian +3
Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environm…
Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen
Chengming Feng, Hesam Araghi, Liming Zheng +4
Event cameras, also known as neuromorphic cameras, have gained significant attention in recent years due to their high temporal resolution, high dynamic range, and low power consum…
Fine-grained Human Motion Understanding with Language Models
Thomas Markhorst, Zhi-Yi Lin, Jouh Yeong Chew +2
Fine-grained motion understanding requires knowing not only which poses occur, but also when they occur. Existing motion-language models either compress motion clips or concatenate…
AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination
Mingyu Han, Hyunyoung Han, Nitheekulawatn Thommakoon +4
Eye tracking is essential for smart glasses, as it provides insight into user attention for ambient intelligence applications. However, most existing eye-tracking systems rely on a…
PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interaction
Zhi-Yi Lin, Thomas Markhorst, Jouh Yeong Chew +1
Human-like multimodal reaction generation is essential for natural group interactions between humans and embodied AI. However, existing approaches are limited to single-modality or…
MuPPet: Multi-person 2D-to-3D Pose Lifting
Thomas Markhorst, Zhi-Yi Lin, Jouh Yeong Chew +2
Multi-person social interactions are inherently built on coherence and relationships among all individuals within the group, making multi-person localization and body pose estimati…
GGPT: Geometry Grounded Point Transformer
Yutong Chen, Yiming Wang, Xucong Zhang +2
Recent feed-forward networks have achieved remarkable progress in sparse-view 3D reconstruction by predicting dense point maps directly from RGB images. However, they often suffer…
Talk to Me, Not the Slides: A Real-Time Wearable Assistant for Improving Eye Contact in Presentations
Lingyu Du, Xucong Zhang, Guohao Lan
Effective eye contact is a cornerstone of successful public speaking. It strengthens the speaker's credibility and fosters audience engagement. Yet, managing effective eye contact…
Gaze-Guided 3D Hand Motion Prediction for Detecting Intent in Egocentric Grasping Tasks
Yufei He, Xucong Zhang, Arno H. A. Stienen
Human intention detection with hand motion prediction is critical to drive the upper-extremity assistive robots in neurorehabilitation applications. However, the traditional method…
UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training
Jiawei Qin, Xucong Zhang, Yusuke Sugano
Despite decades of research on data collection and model architectures, current gaze estimation models encounter significant challenges in generalizing across diverse data domains.…
PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking Services
Lingyu Du, Jinyuan Jia, Xucong Zhang +1
Eye gaze contains rich information about human attention and cognitive processes. This capability makes the underlying technology, known as gaze tracking, a critical enabler for ma…
GazeHTA: End-to-end Gaze Target Detection with Head-Target Association
Zhi-Yi Lin, Jouh Yeong Chew, Jan van Gemert +1
Precisely detecting which object a person is paying attention to is critical for human-robot interaction since it provides important cues for the next action from the human user. W…
3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data
Zhi-Yi Lin, Bofan Lyu, Judith Cueto Fernandez +3
Accurate 3D kinematics estimation of human body is crucial in various applications for human health and mobility, such as rehabilitation, injury prevention, and diagnosis, as it he…
Unsupervised Gaze-aware Contrastive Learning with Subject-specific Condition
Lingyu Du, Xucong Zhang, Guohao Lan
Appearance-based gaze estimation has shown great promise in many applications by using a single general-purpose camera as the input device. However, its success is highly depending…
Investigation of Architectures and Receptive Fields for Appearance-based Gaze Estimation
Yunhan Wang, Xiangwei Shi, Shalini De Mello +2
With the rapid development of deep learning technology in the past decade, appearance-based gaze estimation has attracted great attention from both computer vision and human-comput…
EFE: End-to-end Frame-to-Gaze Estimation
Haldun Balim, Seonwook Park, Xi Wang +2
Despite the recent development of learning-based gaze estimation methods, most methods require one or more eye or face region crops as inputs and produce a gaze direction vector as…
Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement
Jiawei Qin, Takuru Shimoyama, Xucong Zhang +1
Along with the recent development of deep neural networks, appearance-based gaze estimation has succeeded considerably when training and testing within the same domain. Compared to…
Towards Single Camera Human 3D-Kinematics
Marian Bittner, Wei-Tse Yang, Xucong Zhang +3
Markerless estimation of 3D Kinematics has the great potential to clinically diagnose and monitor movement disorders without referrals to expensive motion capture labs; however, cu…
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
Alessandro Ruzzi, Xiangwei Shi, Xi Wang +5
We propose GazeNeRF, a 3D-aware method for the task of gaze redirection. Existing gaze redirection methods operate on 2D images and struggle to generate 3D consistent results. Inst…
PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning
Adrian Spurr, Aneesh Dahiya, Xi Wang +2
Encouraged by the success of contrastive learning on image classification tasks, we propose a new self-supervised method for the structured regression task of 3D hand pose estimati…
Self-Learning Transformations for Improving Gaze and Head Redirection
Yufeng Zheng, Seonwook Park, Xucong Zhang +2
Many computer vision tasks rely on labeled data. Rapid progress in generative modeling has led to the ability to synthesize photorealistic images. However, controlling specific asp…
ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation
Xucong Zhang, Seonwook Park, Thabo Beeler +3
Gaze estimation is a fundamental task in many applications of computer vision, human computer interaction and robotics. Many state-of-the-art methods are trained and tested on cust…
Towards End-to-end Video-based Eye-Tracking
Seonwook Park, Emre Aksan, Xucong Zhang +1
Estimating eye-gaze from images alone is a challenging task, in large parts due to un-observable person-specific factors. Achieving high accuracy typically requires labeled data fr…
Content-Consistent Generation of Realistic Eyes with Style
Marcel Bühler, Seonwook Park, Shalini De Mello +2
Accurately labeled real-world training data can be scarce, and hence recent works adapt, modify or generate images to boost target datasets. However, retaining relevant details fro…