32 citations · 83 across the 6 of their papers we have counts for
7 papers
Gaze-enhanced Crossmodal Embeddings for Emotion Recognition
Ahmed Abdou, Ekta Sood, Philipp Müller +1
Emotional expressions are inherently multimodal -- integrating facial behavior, speech, and gaze -- but their automatic recognition is often limited to a single modality, e.g. spee…
Multimodal Integration of Human-Like Attention in Visual Question Answering
Ekta Sood, Fabian Kögel, Philipp Müller +3
Human-like attention as a supervisory signal to guide neural attention has shown significant promise but is currently limited to uni-modal integration - even for inherently multimo…
VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answering
Ekta Sood, Fabian Kögel, Florian Strohm +2
We present VQA-MHUG - a novel 49-participant dataset of multimodal human gaze on both images and questions during visual question answering (VQA) collected using a high-speed eye t…
Neural Photofit: Gaze-based Mental Image Reconstruction
Florian Strohm, Ekta Sood, Sven Mayer +3
We propose a novel method that leverages human fixations to visually decode the image a person has in mind into a photofit (facial composite). Our method combines three neural netw…
Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention
Ekta Sood, Simon Tannert, Philipp Mueller +1
A lack of corpora has so far limited advances in integrating human gaze data as a supervisory signal in neural attention mechanisms for natural language processing(NLP). We propose…
Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
Ekta Sood, Simon Tannert, Diego Frassinelli +2
While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention r…
Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehension
Matthias Blohm, Glorianna Jagfeld, Ekta Sood +2
We propose a machine reading comprehension model based on the compare-aggregate framework with two-staged attention that achieves state-of-the-art results on the MovieQA question a…