1 citations · 1 across the 11 of their papers we have counts for
11 papers
MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions
Lijian Lin, Ye Zhu, Fan Zhang +5
% !TEX root = ../main.tex Real-time embodied companion interaction requires a robot to infer user intent from streaming speech, generate timely responses, and execute expressive, i…
OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics
Mingxian Lin, Shengju Qian, Yuqi Liu +9
Vision-language model (VLM) agents are increasingly deployed in interactive game environments. Yet game benchmarks for VLM agents typically report a single first-attempt score per…
Your Model Already Knows: Attention-Guided Safety Filter for Vision-Language-Action Models
Seongbin Park, Fan Zhang, Baharan Mirzasoleiman +2
Vision-Language-Action (VLA) models have demonstrated impressive end-to-end performance across a variety of robotic manipulation tasks. However, these policies offer no guarantees…
ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models
Fan Zhang, Seongbin Park, Baharan Mirzasoleiman +2
Vision-Language-Action (VLA) models demonstrate strong perfor-1 mance on language-conditioned robotic manipulation within their training dis-2 tribution, yet their generalization c…
MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction
Joerg Deigmoeller, Nakul Agarwal, Stephan Hasler +8
We introduce MERGE, a system for situational grounding of actors, objects, and events in dynamic human-robot group interactions. Effective collaboration in such settings requires c…
RAE-NWM: Navigation World Model in Dense Visual Representation Space
Mingkun Zhang, Wangtian Shen, Fan Zhang +3
Visual navigation requires agents to reach goals in complex environments through perception and planning. World models address this task by simulating action-conditioned state tran…
Learning Robot Manipulation from Audio World Models
Fan Zhang, Michael Gienger
World models have demonstrated impressive performance on robotic learning tasks. Many such tasks inherently demand multimodal reasoning; for example, filling a bottle with water wi…
Generation of Real-time Robotic Emotional Expressions Learning from Human Demonstration in Mixed Reality
Chao Wang, Michael Gienger, Fan Zhang
Expressive behaviors in robots are critical for effectively conveying their emotional states during interactions with humans. In this work, we present a framework that autonomously…
CARMA: Context-Aware Situational Grounding of Human-Robot Group Interactions by Combining Vision-Language Models with Object and Action Recognition
Joerg Deigmoeller, Stephan Hasler, Nakul Agarwal +8
We introduce CARMA, a system for situational grounding in human-robot group interactions. Effective collaboration in such group settings requires situational awareness based on a c…
CCDP: Composition of Conditional Diffusion Policies with Guided Sampling
Amirreza Razmjoo, Sylvain Calinon, Michael Gienger +1
Imitation Learning offers a promising approach to learn directly from data without requiring explicit models, simulations, or detailed task definitions. During inference, actions a…
Affordance-based Robot Manipulation with Flow Matching
Fan Zhang, Michael Gienger
We present a framework for assistive robot manipulation, which focuses on two fundamental challenges: first, efficiently adapting large-scale models to downstream scene affordance…