25 citations · 54 across the 14 of their papers we have counts for
7 papers · 1 filter
Proactive Hearing Assistants that Isolate Egocentric Conversations
Guilin Hu, Malek Itani, Tuochao Chen +1
We introduce proactive hearing assistants that automatically identify and separate the wearer's conversation partners, without requiring explicit prompts. Our system operates on eg…
AV-Dialog: Spoken Dialogue Models with Audio-Visual Input
Tuochao Chen, Bandhav Veluri, Hongyu Gong +1
Dialogue models falter in noisy, multi-speaker environments, often producing irrelevant responses and awkward turn-taking. We present AV-Dialog, the first multimodal dialog framewo…
TF-MLPNet: Tiny Real-Time Neural Speech Separation
Malek Itani, Tuochao Chen, Shyamnath Gollakota
Speech separation on hearable devices can enable transformative augmented and enhanced hearing capabilities. However, state-of-the-art speech separation networks cannot run in real…
SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction
Tuochao Chen, D Shin, Hakan Erdogan +1
This paper introduces SoundSculpt, a neural network designed to extract target sound fields from ambisonic recordings. SoundSculpt employs an ambisonic-in-ambisonic-out architectur…
LLAMAPIE: Proactive In-Ear Conversation Assistants
Tuochao Chen, Nicholas Batchelder, Alisa Liu +2
We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike tra…
Spatial Speech Translation: Translating Across Space With Binaural Hearables
Tuochao Chen, Qirui Wang, Runlin He +1
Imagine being in a crowded space where people speak a different language and having hearables that transform the auditory space into your native language, while preserving the spat…
Wireless Hearables With Programmable Speech AI Accelerators
Malek Itani, Tuochao Chen, Arun Raghavan +2
The conventional wisdom has been that designing ultra-compact, battery-constrained wireless hearables with on-device speech AI models is challenging due to the high computational d…