12 papers
Optimal Transport Q-Learning for Flow Policy Steering and Acceleration
Andreas Sochopoulos, Esmeralda S. Whitammer, Nikolaos Tsagkas +3
Diffusion and flow policies have recently demonstrated remarkable performance in robotic applications by accurately capturing multimodal robot trajectory distributions, especially…
SemanticScanpath: Combining Gaze and Speech for Situated Human-Robot Interaction Using LLMs
Elisabeth Menendez, Michael Gienger, Santiago MartÃnez +2
Large Language Models (LLMs) have substantially improved the conversational capabilities of social robots. Nevertheless, for an intuitive and fluent human-robot interaction, robots…
MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction
Joerg Deigmoeller, Nakul Agarwal, Stephan Hasler +8
We introduce MERGE, a system for situational grounding of actors, objects, and events in dynamic human-robot group interactions. Effective collaboration in such settings requires c…
XR: An Extended Reality Platform for Social-Physical Human-Robot Interaction
Chao Wang, Anna Belardinelli, Michael Gienger
Social-physical human-robot interaction (spHRI) is difficult to study: building and programming robots that integrate multiple interaction modalities is costly and slow, while VR-b…
Generation of Real-time Robotic Emotional Expressions Learning from Human Demonstration in Mixed Reality
Chao Wang, Michael Gienger, Fan Zhang
Expressive behaviors in robots are critical for effectively conveying their emotional states during interactions with humans. In this work, we present a framework that autonomously…
Learning Robot Manipulation from Audio World Models
Fan Zhang, Michael Gienger
World models have demonstrated impressive performance on robotic learning tasks. Many such tasks inherently demand multimodal reasoning; for example, filling a bottle with water wi…