4 papers
A Multimodal Depth-Aware Method For Embodied Reference Understanding
Fevziye Irem Eyiokur, Dogucan Yaman, Hazım Kemal Ekenel +1
Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While prior works have shown progress…
Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Framework
Dogucan Yaman, Fevziye Irem Eyiokur, Hazım Kemal Ekenel +1
Video editing-based talking face generation aims to preserve video details such as pose, lighting, and gestures while modifying only lip motion, often using an identity reference i…
CAPE: A CLIP-Aware Pointing Ensemble of Complementary Heatmap Cues for Embodied Reference Understanding
Fevziye Irem Eyiokur, Dogucan Yaman, Hazım Kemal Ekenel +1
We address Embodied Reference Understanding, the task of predicting the object a person in the scene refers to through pointing gesture and language. This requires multimodal reaso…
Shared Latent Representation for Joint Text-to-Audio-Visual Synthesis
Dogucan Yaman, Seymanur Akti, Fevziye Irem Eyiokur +1
We propose a text-to-talking-face synthesis framework leveraging latent speech representations from HierSpeech++. A Text-to-Vec module generates Wav2Vec2 embeddings from text, whic…