8 citations · 21 across the 16 of their papers we have counts for
17 papers
SPHERE: Automatic Music Upmixing via Audio Language Model Post-Training with Spatial Heuristic Rewards
Zixun Guo, Calvin Murdock, Sanjeel Parekh +4
In this paper, we study the task of automatic music upmixing, wherein a system predicts spatial mixing parameters from a multi-stem recording. Different from existing methods that…
Ambisonics Encoding of Room Impulse Responses using a Device-Agnostic Diffusion Model
Eloi Moliner, Christoph Hold, Juan Azcarreta Ortiz +5
We address the problem of encoding room impulse responses (RIRs) into high-order Ambisonics (HOA) representations from arbitrary and potentially insufficient or incomplete micropho…
Towards Localizing Conversation Partners using Head Motion
Payal Mohapatra, Calvin Murdock, Ali Aroudi +4
Many individuals struggle to understand conversation partners in noisy settings, particularly amid background speakers or due to hearing impairments. Emerging wearables like smartg…
Text-to-Stage: Spatial Layouts from Long-form Narratives
Jefferson Hernandez, Swarnadeep Saha, Chenxi Whitehouse +6
In this work, we probe the ability of a language model to demonstrate spatial reasoning from unstructured text, mimicking human capabilities and automating a process that benefits…
Towards Perception-Informed Latent HRTF Representations
You Zhang, Andrew Francl, Ruohan Gao +3
Personalized head-related transfer functions (HRTFs) are essential for ensuring a realistic auditory experience over headphones, because they take into account individual anatomica…
EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception
Sanjoy Chowdhury, Subrata Biswas, Sayan Nag +7
Modern perception models, particularly those designed for multisensory egocentric tasks, have achieved remarkable performance but often come with substantial computational costs. T…
SoundVista: Novel-View Ambient Sound Synthesis via Visual-Acoustic Binding
Mingfei Chen, Israel D. Gebru, Ishwarya Ananthabhotla +7
We introduce SoundVista, a method to generate the ambient sound of an arbitrary scene at novel viewpoints. Given a pre-acquired recording of the scene from sparsely distributed mic…
Hearing Anywhere in Any Environment
Xiulong Liu, Anurag Kumar, Paul Calamia +7
In mixed reality applications, a realistic acoustic experience in spatial environments is as crucial as the visual experience for achieving true immersion. Despite recent advances…
Scene-wide Acoustic Parameter Estimation
Ricardo Falcon-Perez, Ruohan Gao, Gregor Mueckl +2
For augmented (AR) and virtual reality (VR) applications, accurate estimates of the acoustic characteristics of a scene are critical for creating a sense of immersion. However, dir…
Spherical World-Locking for Audio-Visual Localization in Egocentric Videos
Heeseung Yun, Ruohan Gao, Ishwarya Ananthabhotla +6
Egocentric videos provide comprehensive contexts for user and scene understanding, spanning multisensory perception to behavioral interaction. We propose Spherical World-Locking (S…
On HRTF Notch Frequency Prediction Using Anthropometric Features and Neural Networks
Lior Arbel, Ishwarya Ananthabhotla, Zamir Ben-Hur +2
High fidelity spatial audio often performs better when produced using a personalized head-related transfer function (HRTF). However, the direct acquisition of HRTFs is cumbersome a…
Hearing Loss Detection from Facial Expressions in One-on-one Conversations
Yufeng Yin, Ishwarya Ananthabhotla, Vamsi Krishna Ithapu +3
Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captu…
The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective
Wenqi Jia, Miao Liu, Hao Jiang +4
In recent years, the thriving development of research related to egocentric videos has provided a unique perspective for the study of conversational interactions, where both visual…
Towards Improved Room Impulse Response Estimation for Speech Recognition
Anton Ratnarajah, Ishwarya Ananthabhotla, Vamsi Krishna Ithapu +3
We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of a downstream application scenario, far-field automatic speech recognition (AS…
The Intrinsic Memorability of Everyday Sounds
David B. Ramsay, Ishwarya Ananthabhotla, Joseph A. Paradiso
Our aural experience plays an integral role in the perception and memory of the events in our lives. Some of the sounds we encounter throughout the day stay lodged in our minds mor…
SoundSignaling: Realtime, Stylistic Modification of a Personal Music Corpus for Information Delivery
Ishwarya Ananthabhotla, Joseph A. Paradiso
Drawing inspiration from the notion of cognitive incongruence associated with Stroop's famous experiment, from musical principles, and from the observation that music consumption o…
HCU400: An Annotated Dataset for Exploring Aural Phenomenology Through Causal Uncertainty
Ishwarya Ananthabhotla, David B. Ramsay, Joseph A. Paradiso
The way we perceive a sound depends on many aspects-- its ecological frequency, acoustic features, typicality, and most notably, its identified source. In this paper, we present th…