256 citations · 1.5k across the 113 of their papers we have counts for
43 papers · 1 filter
Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling
Virmarie Maquiling, Zhuojiang Cai, Enkelejda Kasneci
Gaze is increasingly used as an input signal for vision and multimodal models, yet no consensus exists on how to represent it across datasets. Raw traces preserve detail but are no…
An Affordable, Wearable Stereo-Eye-Tracking Platform
Alexander Zimmer, Yasmeen Abdrabou, Enkelejda Kasneci
Research on video-based eye-tracking has long explored stereo and glint-based methods, yet existing wearable eye trackers - both commercial and open-source - offer limited flexibil…
Night Eyes: A Reproducible Framework for Constellation-Based Corneal Reflection Matching
Virmarie Maquiling, Yasmeen Abdrabou, Enkelejda Kasneci
Corneal reflection (glint) detection plays an important role in pupil-corneal reflection (P-CR) eye tracking, but in practice it is often handled as heuristics embedded within larg…
Exploring Automated Recognition of Instructional Activity and Discourse from Multimodal Classroom Data
Ivo Bueno, Ruikun Hou, Babette Bühler +7
Observation of classroom interactions can provide concrete feedback to teachers, but current methods rely on manual annotation, which is resource-intensive and hard to scale. This…
Cross-View Cross-Modal Unsupervised Domain Adaptation for Driver Monitoring System
Aditi Bhalla, Christian Hellert, Enkelejda Kasneci
Driver distraction remains a leading cause of road traffic accidents, contributing to thousands of fatalities annually across the globe. While deep learning-based driver activity r…
Automated Visual Attention Detection using Mobile Eye Tracking in Behavioral Classroom Studies
Efe Bozkir, Christian Kosel, Tina Seidel +1
Teachers' visual attention and its distribution across the students in classrooms can constitute important implications for student engagement, achievement, and professional teache…
Trade-offs in Privacy-Preserving Eye Tracking through Iris Obfuscation: A Benchmarking Study
Mengdi Wang, Efe Bozkir, Enkelejda Kasneci
Recent developments in hardware, computer graphics, and AI may soon enable AR/VR head-mounted displays (HMDs) to become everyday devices like smartphones and tablets. Eye trackers…
Iris Style Transfer: Enhancing Iris Recognition with Style Features and Privacy Preservation through Neural Style Transfer
Mengdi Wang, Efe Bozkir, Enkelejda Kasneci
Iris texture is widely regarded as a gold standard biometric modality for authentication and identification. The demand for robust iris recognition methods, coupled with growing se…
OCDet: Object Center Detection via Bounding Box-Aware Heatmap Prediction on Edge Devices with NPUs
Chen Xin, Thomas Motz, Andreas Hartel +1
Real-time object localization on edge devices is fundamental for numerous applications, ranging from surveillance to industrial automation. Traditional frameworks, such as object d…
Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images
Virmarie Maquiling, Sean Anthony Byrne, Diederick C. Niehorster +2
We explore the transformative potential of SAM 2, a vision foundation model, in advancing gaze estimation and eye tracking technologies. By significantly reducing annotation time,…
DART: An Automated End-to-End Object Detection Pipeline with Data Diversification, Open-Vocabulary Bounding Box Annotation, Pseudo-Label Review, and Model Training
Chen Xin, Andreas Hartel, Enkelejda Kasneci
Accurate real-time object detection is vital across numerous industrial applications, from safety monitoring to quality control. Traditional approaches, however, are hindered by ar…
Pretrained Visual Uncertainties
Michael Kirchhof, Mark Collier, Seong Joon Oh +1
Accurate uncertainty estimation is vital to trustworthy machine learning, yet uncertainties typically have to be learned for each task anew. This work introduces the first pretrain…
Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)
Virmarie Maquiling, Sean Anthony Byrne, Diederick C. Niehorster +2
The advent of foundation models signals a new era in artificial intelligence. The Segment Anything Model (SAM) is the first foundation model for image segmentation. In this study,…
LEyes: A Lightweight Framework for Deep Learning-Based Eye Tracking using Synthetic Eye Images
Sean Anthony Byrne, Virmarie Maquiling, Marcus Nyström +2
Deep learning has bolstered gaze estimation techniques, but real-world deployment has been impeded by inadequate training datasets. This problem is exacerbated by both hardware-ind…
DynStatF: An Efficient Feature Fusion Strategy for LiDAR 3D Object Detection
Yao Rong, Xiangyu Wei, Tianwei Lin +2
Augmenting LiDAR input with multiple previous frames provides richer semantic information and thus boosts performance in 3D object detection, However, crowded point clouds in multi…
Precise localization of corneal reflections in eye images using deep learning trained on synthetic data
Sean Anthony Byrne, Marcus Nyström, Virmarie Maquiling +2
We present a deep learning method for accurately localizing the center of a single corneal reflection (CR) in an eye image. Unlike previous approaches, we use a convolutional neura…
Where and What: Driver Attention-based Object Detection
Yao Rong, Naemi-Rebecca Kassautzki, Wolfgang Fuhl +1
Human drivers use their attentional mechanisms to focus on critical objects and make decisions while driving. As human attention can be revealed from gaze data, capturing and analy…
A Consistent and Efficient Evaluation Strategy for Attribution Methods
Yao Rong, Tobias Leemann, Vadim Borisov +2
With a variety of local feature attribution methods being proposed in recent years, follow-up work suggested several evaluation strategies. To assess the attribution quality across…
Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gaze
Wolfgang Fuhl, Daniel Weber, Shahram Eivazi
This paper describes a feature extraction and gaze estimation software, named \textit{Pistol} that can be used with Pupil Invisible projects and other eye trackers in the future. I…
Human Attention in Fine-grained Classification
Yao Rong, Wenjia Xu, Zeynep Akata +1
The way humans attend to, process and classify a given image has the potential to vastly benefit the performance of deep learning models. Exploiting where humans are focusing can r…
Estimating Presentation Competence using Multimodal Nonverbal Behavioral Cues
Ömer Sümer, Cigdem Beyan, Fabian Ruth +3
Public speaking and presentation competence plays an essential role in many areas of social interaction in our educational, professional, and everyday life. Since our intention dur…
Differentiating Surgeon Expertise Solely by Eye Movement Features
Benedikt Hosp, Myat Su Yin, Peter Haddawy +2
Developments in computer science in recent years are moving into hospitals. Surgeons are faced with ever new technical challenges. Visual perception plays a key role in most of the…
Multimodal Engagement Analysis from Facial Videos in the Classroom
Ömer Sümer, Patricia Goldberg, Sidney D'Mello +3
Student engagement is a key construct for learning and teaching. While most of the literature explored the student engagement analysis on computer-based settings, this paper extend…
A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis
Wolfgang Fuhl, Enkelejda Kasneci
We present a new dataset with annotated eye movements. The dataset consists of over 800,000 gaze points recorded during a car ride in the real world and in the simulator. In total,…