54 citations · 114 across the 17 of their papers we have counts for
5 papers · 1 filter
Learning Joint Representation of Human Motion and Language
Jihoon Kim, Youngjae Yu, Seungyoun Shin +2
In this work, we present MoLang (a Motion-Language connecting model) for learning joint representation of human motion and language, leveraging both unpaired and paired datasets of…
Zero-shot Active Visual Search (ZAVIS): Intelligent Object Search for Robotic Assistants
Jeongeun Park, Taerim Yoon, Jejoon Hong +3
In this paper, we focus on the problem of efficiently locating a target object described with free-form language using a mobile robot equipped with vision sensors (e.g., an RGBD ca…
Multimodal Knowledge Alignment with Reinforcement Learning
Youngjae Yu, Jiwan Chung, Heeseung Yun +8
Large language models readily adapt to novel settings, even without task-specific training data. Can their zero-shot capacity be extended to multimodal inputs? In this work, we pro…
ProsocialDialog: A Prosocial Backbone for Conversational Agents
Hyunwoo Kim, Youngjae Yu, Liwei Jiang +5
Most existing dialogue systems fail to respond properly to potentially unsafe user utterances by either ignoring or passively agreeing with them. To address this issue, we introduc…
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound
Rowan Zellers, Jiasen Lu, Ximing Lu +7
As humans, we navigate a multimodal world, building a holistic understanding from all our senses. We introduce MERLOT Reserve, a model that represents videos jointly over time -- t…