8 papers
Fine-grained Soundscape Control for Augmented Hearing
Seunghyun Oh, Malek Itani, Aseem Gauri +1
Hearables are becoming ubiquitous, yet their sound controls remain blunt: users can either enable global noise suppression or focus on a single target sound. Real-world acoustic sc…
Proactive Hearing Assistants that Isolate Egocentric Conversations
Guilin Hu, Malek Itani, Tuochao Chen +1
We introduce proactive hearing assistants that automatically identify and separate the wearer's conversation partners, without requiring explicit prompts. Our system operates on eg…
AV-Dialog: Spoken Dialogue Models with Audio-Visual Input
Tuochao Chen, Bandhav Veluri, Hongyu Gong +1
Dialogue models falter in noisy, multi-speaker environments, often producing irrelevant responses and awkward turn-taking. We present AV-Dialog, the first multimodal dialog framewo…
Wireless Hearables With Programmable Speech AI Accelerators
Malek Itani, Tuochao Chen, Arun Raghavan +2
The conventional wisdom has been that designing ultra-compact, battery-constrained wireless hearables with on-device speech AI models is challenging due to the high computational d…
TF-MLPNet: Tiny Real-Time Neural Speech Separation
Malek Itani, Tuochao Chen, Shyamnath Gollakota
Speech separation on hearable devices can enable transformative augmented and enhanced hearing capabilities. However, state-of-the-art speech separation networks cannot run in real…
Neural Speech Extraction with Human Feedback
Malek Itani, Ashton Graves, Sefik Emre Eskimez +1
We present the first neural target speech extraction (TSE) system that uses human feedback for iterative refinement. Our approach allows users to mark specific segments of the TSE…