15 citations · 18 across the 11 of their papers we have counts for
13 papers
Understanding and Reducing Misalignment in Persuasion Optimization
Boyi Wei, Ruta Desai, Peter Henderson +1
Frontier models are now trained for complex multi-agent tasks, from running a company to engaging in negotiations. These tasks necessitate training models to be persuasive. But wha…
Learning to Interrupt in Language-based Multi-agent Communication
Danqing Wang, Da Yin, Ruta Desai +3
When a colleague starts explaining something you already understand, you interrupt them. This simple act, a listener taking control of the conversation, is natural in human communi…
COOPERA: Continual Open-Ended Human-Robot Assistance
Chenyang Ma, Kai Lu, Ruta Desai +3
To understand and collaborate with humans, robots must account for individual human traits, habits, and activities over time. However, most robotic assistants lack these abilities,…
Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations
Edward Kim, Daniel He, Jorge Chao +7
Teaching systems physical tasks is a long standing goal in HCI, yet most prior work has focused on non collaborative physical activities. Collaborative tasks introduce added comple…
Enhancing Visual Planning with Auxiliary Tasks and Multi-token Prediction
Ce Zhang, Yale Song, Ruta Desai +4
Visual Planning for Assistance (VPA) aims to predict a sequence of user actions required to achieve a specified goal based on a video showing the user's progress. Although recent a…
ADAPT: Actively Discovering and Adapting to Preferences for any Task
Maithili Patel, Xavier Puig, Ruta Desai +4
Assistive agents should be able to perform under-specified long-horizon tasks while respecting user preferences. We introduce Actively Discovering and Adapting to Preferences for a…
Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning
Ram Ramrakhya, Matthew Chang, Xavier Puig +3
Embodied agents operating in household environments must interpret ambiguous and under-specified human instructions. A capable household robot should recognize ambiguity and ask re…
PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
Matthew Chang, Gunjan Chhablani, Alexander Clegg +17
We present a benchmark for Planning And Reasoning Tasks in humaN-Robot collaboration (PARTNR) designed to study human-robot coordination in household activities. PARTNR tasks exhib…
Human Action Anticipation: A Survey
Bolin Lai, Sam Toyer, Tushar Nagarajan +7
Predicting future human behavior is an increasingly popular topic in computer vision, driven by the interest in applications such as autonomous vehicles, digital assistants and hum…
User-in-the-loop Evaluation of Multimodal LLMs for Activity Assistance
Mrinal Verghese, Brian Chen, Hamid Eghbalzadeh +2
Our research investigates the capability of modern multimodal reasoning models, powered by Large Language Models (LLMs), to facilitate vision-powered assistants for multi-step dail…
Learning Human Preferences Over Robot Behavior as Soft Planning Constraints
Austin Narcomey, Nathan Tsoi, Ruta Desai +1
Preference learning has long been studied in Human-Robot Interaction (HRI) in order to adapt robot behavior to specific user needs and desires. Typically, human preferences are mod…
Human-Centered Planning
Yuliang Li, Nitin Kamra, Ruta Desai +1
LLMs have recently made impressive inroads on tasks whose output is structured, such as coding, robotic planning and querying databases. The vision of creating AI-powered personal…
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
Xavier Puig, Eric Undersander, Andrew Szot +20
We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accur…