8 papers
A Multimodal Depth-Aware Method For Embodied Reference Understanding
Fevziye Irem Eyiokur, Dogucan Yaman, Hazım Kemal Ekenel +1
Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While prior works have shown progress…
Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Framework
Dogucan Yaman, Fevziye Irem Eyiokur, Hazım Kemal Ekenel +1
Video editing-based talking face generation aims to preserve video details such as pose, lighting, and gestures while modifying only lip motion, often using an identity reference i…
CAPE: A CLIP-Aware Pointing Ensemble of Complementary Heatmap Cues for Embodied Reference Understanding
Fevziye Irem Eyiokur, Dogucan Yaman, Hazım Kemal Ekenel +1
We address Embodied Reference Understanding, the task of predicting the object a person in the scene refers to through pointing gesture and language. This requires multimodal reaso…
Shared Latent Representation for Joint Text-to-Audio-Visual Synthesis
Dogucan Yaman, Seymanur Akti, Fevziye Irem Eyiokur +1
We propose a text-to-talking-face synthesis framework leveraging latent speech representations from HierSpeech++. A Text-to-Vec module generates Wav2Vec2 embeddings from text, whic…
Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation
Dogucan Yaman, Fevziye Irem Eyiokur, Leonard Bärmann +2
Audio-Driven Talking Face Generation aims at generating realistic videos of talking faces, focusing on accurate audio-lip synchronization without deteriorating any identity-related…
Titanic Calling: Low Bandwidth Video Conference from the Titanic Wreck
Fevziye Irem Eyiokur, Christian Huber, Thai-Binh Nguyen +5
In this paper, we report on communication experiments conducted in the summer of 2022 during a deep dive to the wreck of the Titanic. Radio transmission is not possible in deep sea…