4 papers
RESPOND: Responsive Engagement Strategy for Predictive Orchestration and Dialogue
Meng-Chen Lee, Costas Panay, Javier Hernandez +4
The majority of voice-based conversational agents still rely on pause-and-respond turn-taking, leaving interactions sounding stiff and robotic. We present RESPOND (Responsive Engag…
SigmaCollab: An Application-Driven Dataset for Physically Situated Collaboration
Dan Bohus, Sean Andrist, Ann Paradiso +3
We introduce SigmaCollab, a dataset enabling research on physically situated human-AI collaboration. The dataset consists of a set of 85 sessions in which untrained participants we…
"Is This It?": Towards Ecologically Valid Benchmarks for Situated Collaboration
Dan Bohus, Sean Andrist, Yuwei Bao +2
We report initial work towards constructing ecologically valid benchmarks to assess the capabilities of large multimodal models for engaging in situated collaboration. In contrast…
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research
Dan Bohus, Sean Andrist, Nick Saw +3
We introduce an open-source system called SIGMA (short for "Situated Interactive Guidance, Monitoring, and Assistance") as a platform for conducting research on task-assistive agen…