6 papers
Real-time Speech Restoration using Data Prediction Mean Flows
Sebastian Braun
Generative models are capable to address difficult problems with non-unique solutions like bandwidth extension and gap filling, removing highly non-linear artifacts from codecs, cl…
FOA Tokenizer: Low-bitrate Neural Codec for First Order Ambisonics with Spatial Consistency Loss
Parthasaarathy Sudarsanam, Sebastian Braun, Hannes Gamper
Neural audio codecs have been widely studied for mono and stereo signals, but spatial audio remains largely unexplored. We present the first discrete neural spatial audio codec for…
SALAD-VAE: Semantic Audio Compression with Language-Audio Distillation
Sebastian Braun, Hannes Gamper, Dimitra Emmanouilidou
Modern generative and multimodal models increasingly rely on compact latent representations that trade and balance semantic richness with high-fidelity reconstruction. We introduce…
Sci-Phi: A Large Language Model Spatial Audio Descriptor
Xilin Jiang, Hannes Gamper, Sebastian Braun
Acoustic scene perception involves describing the type of sounds, their timing, their direction and distance, as well as their loudness and reverberation. While audio language mode…
Distributed Asynchronous Device Speech Enhancement via Windowed Cross-Attention
Gene-Ping Yang, Sebastian Braun
The increasing number of microphone-equipped personal devices offers great flexibility and potential using them as ad-hoc microphone arrays in dynamic meeting environments. However…
CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling
Ruihan Yang, Hannes Gamper, Sebastian Braun
We introduce a multi-modal diffusion model tailored for the bi-directional conditional generation of video and audio. We propose a joint contrastive training loss to improve the sy…