25 citations · 26 across the 9 of their papers we have counts for
10 papers
DYAD: A Multimodal Dataset of Co-Located Human Assistance
Akhil Ajikumar, Mahya Qorbani, Sakib Reza +2
An embodied assistant working beside a person must track task state, recognize help seeking, choose how to intervene, and produce an appropriate response. Existing procedural datas…
RESPOND: Responsive Engagement Strategy for Predictive Orchestration and Dialogue
Meng-Chen Lee, Costas Panay, Javier Hernandez +4
The majority of voice-based conversational agents still rely on pause-and-respond turn-taking, leaving interactions sounding stiff and robotic. We present RESPOND (Responsive Engag…
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso +3
We introduce SigmaCollab, a dataset enabling research on physically situated human-AI collaboration. The dataset consists of a set of 85 sessions in which untrained participants we…
"Is This It?": Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Yuwei Bao +2
We report initial work towards constructing ecologically valid benchmarks to assess the capabilities of large multimodal models for engaging in situated collaboration. In contrast…
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research
Dan Bohus, Sean Andrist, Nick Saw +3
We introduce an open-source system called SIGMA (short for "Situated Interactive Guidance, Monitoring, and Assistance") as a platform for conducting research on task-assistive agen…
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
Xin Wang, Taein Kwon, Mahdi Rad +9
Building an interactive AI assistant that can perceive, reason, and collaborate with humans in the real world has been a long-standing pursuit in the AI community. This work is par…
Platform for Situated Intelligence
Dan Bohus, Sean Andrist, Ashley Feniello +5
We introduce Platform for Situated Intelligence, an open-source framework created to support the rapid development and study of multimodal, integrative-AI systems. The framework pr…
Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence
Sean Andrist, Dan Bohus
We describe Platform for Situated Intelligence, an open-source framework for multimodal, integrative-AI systems. The framework provides infrastructure, tools, and components that e…
REFORM: Recognizing F-formations for Social Robots
Hooman Hedayati, Annika Muehlbradt, Daniel J. Szafir +1
Recognizing and understanding conversational groups, or F-formations, is a critical task for situated agents designed to interact with humans. F-formations contain complex structur…
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal +4
Assemblies of modular subsystems are being pressed into service to perform sensing, reasoning, and decision making in high-stakes, time-critical tasks in such areas as transportati…