End-to-end Listen, Look, Speak and Act

Siyin Wang, Wenyi Yu, Xianzhao Chen +4

Human interaction is inherently multimodal and full-duplex: we listen while watching, speak while acting, and fluidly adapt to turn-taking and interruptions. Realizing these capabi…