4 papers
SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays
Sonal Kumar, Sinan Hersek, Artem Dementyev +6
Embodied, ego-centric intelligence fundamentally requires the ability to comprehend spatial audio within complex environments. While large audio-language models excel at mono-chann…
PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs
Artem Dementyev, Wazeer Zulfikar, Sinan Hersek +3
Current multimodal LLMs process audio as a mono stream, ignoring the rich spatial information essential for embodied AI. Existing spatial audio models, conversely, are constrained…
EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses
Akshay Paruchuri, Sinan Hersek, Lavisha Aggarwal +6
All-day smart glasses are likely to emerge as platforms capable of continuous contextual sensing, uniquely positioning them for unprecedented assistance in our daily lives. Integra…
SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction
Tuochao Chen, D Shin, Hakan Erdogan +1
This paper introduces SoundSculpt, a neural network designed to extract target sound fields from ambisonic recordings. SoundSculpt employs an ambisonic-in-ambisonic-out architectur…