165 citations · 619 across the 15 of their papers we have counts for
16 papers
RA-ClipScore: Making Generative Model Evaluation More Interpretable
Yifan Lu, Taras Kucherenko, Hedvig Kjellström +1
Generative models can produce images nearly indistinguishable from real data, yet rigorous and interpretable evaluation remains challenging. Conventional metrics such as FID provid…
Evaluating Data-Driven Co-Speech Gestures of Embodied Conversational Agents through Real-Time Interaction
Yuan He, André Pereira, Taras Kucherenko
Embodied Conversational Agents that make use of co-speech gestures can enhance human-machine interactions in many ways. In recent years, data-driven gesture generation approaches f…
To Rate or Not To Rate: Investigating Evaluation Methods for Generated Co-Speech Gestures
Pieter Wolfert, Jeffrey M. Girard, Taras Kucherenko +1
While automatic performance metrics are crucial for machine learning of artificial human-like behaviour, the gold standard for evaluation remains human judgement. The subjective ev…
Multimodal analysis of the predictability of hand-gesture properties
Taras Kucherenko, Rajmund Nagy, Michael Neff +2
Embodied conversational agents benefit from being able to accompany their speech with gestures. Although many data-driven approaches to gesture generation have been proposed in rec…
Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech
Taras Kucherenko, Rajmund Nagy, Patrik Jonell +3
We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture…
A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents
Rajmund Nagy, Taras Kucherenko, Birger Moell +3
Embodied conversational agents (ECAs) benefit from non-verbal behavior for natural and efficient interaction with users. Gesticulation - hand and arm movements accompanying speech…
A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020
Taras Kucherenko, Patrik Jonell, Youngwoo Yoon +2
Co-speech gestures, gestures that accompany speech, play an important role in human communication. Automatic co-speech gesture generation is thus a key enabling technology for embo…
Generating coherent spontaneous speech and gesture from text
Simon Alexanderson, Éva Székely, Gustav Eje Henter +2
Embodied human communication encompasses both verbal (speech) and non-verbal information (e.g., gesture and head movements). Recent advances in machine learning have substantially…
HEMVIP: Human Evaluation of Multiple Videos in Parallel
Patrik Jonell, Youngwoo Yoon, Pieter Wolfert +2
In many research areas, for example motion and gesture generation, objective measures alone do not provide an accurate impression of key stimulus traits such as perceived quality o…
Can we trust online crowdworkers? Comparing online and offline participants in a preference test of virtual agents
Patrik Jonell, Taras Kucherenko, Ilaria Torre +1
Conducting user studies is a crucial component in many scientific fields. While some studies require participants to be physically present, other studies can be conducted both phys…
Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation
Taras Kucherenko, Dai Hasegawa, Naoshi Kaneko +2
This paper presents a novel framework for speech-driven gesture production, applicable to virtual agents to enhance human-computer interaction. Specifically, we extend recent deep-…
Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings
Patrik Jonell, Taras Kucherenko, Gustav Eje Henter +1
To enable more natural face-to-face interactions, conversational agents need to adapt their behavior to their interlocutors. One key aspect of this is generation of appropriate non…
Gesticulator: A framework for semantically-aware speech-driven gesture generation
Taras Kucherenko, Patrik Jonell, Sanne van Waveren +4
During speech, people spontaneously gesticulate, which plays a key role in conveying information. Similarly, realistic co-speech gestures are crucial to enable natural and smooth i…
Analyzing Input and Output Representations for Speech-Driven Gesture Generation
Taras Kucherenko, Dai Hasegawa, Gustav Eje Henter +2
This paper presents a novel framework for automatic speech-driven gesture generation, applicable to human-agent interaction including both virtual agents and robots. Specifically,…
A Neural Network Approach to Missing Marker Reconstruction in Human Motion Capture
Taras Kucherenko, Jonas Beskow, Hedvig Kjellström
Optical motion capture systems have become a widely used technology in various fields, such as augmented reality, robotics, movie production, etc. Such systems use a large number o…
Machine Learning and Social Robotics for Detecting Early Signs of Dementia
Patrik Jonell, Joseph Mendelson, Thomas Storskog +12
This paper presents the EACare project, an ambitious multi-disciplinary collaboration with the aim to develop an embodied system, capable of carrying out neuropsychological tests t…