Plan, Watch, Recover: A Benchmark and Architectures for Proactive Procedural Assistance

Kaustav Kundu, Ritvik Shrivastava, Maxim Arap +13

We envision a proactive multi-modal assistant system which gives users real-time step-by-step guidance on a procedural task, autonomously deciding \textit{when} to interrupt, and \…