2 citations · 4 across the 6 of their papers we have counts for
6 papers
"Is This It?": Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Yuwei Bao +2
We report initial work towards constructing ecologically valid benchmarks to assess the capabilities of large multimodal models for engaging in situated collaboration. In contrast…
User Modeling Challenges in Interactive AI Assistant Systems
Megan Su, Yuwei Bao
Interactive Artificial Intelligent(AI) assistant systems are designed to offer timely guidance to help human users to complete a variety tasks. One of the remaining challenges is t…
Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake?
Yuwei Bao, Keunwoo Peter Yu, Yichi Zhang +6
Despite tremendous advances in AI, it remains a significant challenge to develop interactive task guidance systems that can offer situated, personalized guidance and assist humans…
Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition
Yuwei Bao, Barrett Martin Lattimer, Joyce Chai
Human language acquisition is an efficient, supervised, and continual process. In this work, we took inspiration from how human babies acquire their first language, and developed a…
DANLI: Deliberative Agent for Following Natural Language Instructions
Yichi Zhang, Jianing Yang, Jiayi Pan +6
Recent years have seen an increasing amount of work on embodied AI agents that can perform tasks by following human language instructions. However, most of these agents are reactiv…
Learning to Mediate Disparities Towards Pragmatic Communication
Yuwei Bao, Sayan Ghosh, Joyce Chai
Human communication is a collaborative process. Speakers, on top of conveying their own intent, adjust the content and language expressions by taking the listeners into account, in…