116 citations · 349 across the 21 of their papers we have counts for
31 papers
GASS: Generalizing Audio Source Separation with Large-scale Data
Jordi Pons, Xiaoyu Liu, Santiago Pascual +1
Universal source separation targets at separating the audio sources of an arbitrary mix, removing the constraint to operate on a specific domain like speech or music. Yet, the pote…
CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Hao-Wen Dong, Xiaoyu Liu, Jordi Pons +5
Recent work has studied text-to-audio synthesis using large amounts of paired text-audio data. However, audio recordings with high-quality text annotations can be difficult to acqu…
Mono-to-stereo through parametric stereo generation
Joan Serrà, Davide Scaini, Santiago Pascual +4
Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific pa…
Towards Robust Image-in-Audio Deep Steganography
Jaume Ros, Margarita Geleta, Jordi Pons +1
The field of steganography has experienced a surge of interest due to the recent advancements in AI-powered techniques, particularly in the context of multimodal setups that enable…
Full-band General Audio Synthesis with Score-based Diffusion
Santiago Pascual, Gautam Bhattacharya, Chunghsin Yeh +2
Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental s…
Adversarial Permutation Invariant Training for Universal Sound Separation
Emilian Postolache, Jordi Pons, Santiago Pascual +1
Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models t…
PodcastMix: A dataset for separating music and speech in podcasts
Nicolás Schmidt, Jordi Pons, Marius Miron
We introduce PodcastMix, a dataset formalizing the task of separating background music and foreground speech in podcasts. We aim at defining a benchmark suitable for training and e…
Universal Speech Enhancement with Score-based Diffusion
Joan Serrà, Santiago Pascual, Jordi Pons +2
Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings.…
On loss functions and evaluation metrics for music source separation
Enric Gusó, Jordi Pons, Santiago Pascual +1
We investigate which loss functions provide better separations via benchmarking an extensive set of those for music source separation. To that end, we first survey the most represe…
Upsampling layers for music source separation
Jordi Pons, Joan Serrà, Santiago Pascual +3
Upsampling artifacts are caused by problematic upsampling layers and due to spectral replicas that emerge while upsampling. Also, depending on the used upsampling layer, such artif…
Adversarial Auto-Encoding for Packet Loss Concealment
Santiago Pascual, Joan Serrà, Jordi Pons
Communication technologies like voice over IP operate under constrained real-time conditions, with voice packets being subject to delays and losses from the network. In such cases,…
PixInWav: Residual Steganography for Hiding Pixels in Audio
Margarita Geleta, Cristina Punti, Kevin McGuinness +3
Steganography comprises the mechanics of hiding data in a host media that may be publicly available. While previous works focused on unimodal setups (e.g., hiding images in images,…
On tuning consistent annealed sampling for denoising score matching
Joan Serrà, Santiago Pascual, Jordi Pons
Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized…
Multichannel-based learning for audio object extraction
Daniel Arteaga, Jordi Pons
The current paradigm for creating and deploying immersive audio content is based on audio objects, which are composed of an audio track and position metadata. While rendering an ob…
On permutation invariant training for speech source separation
Xiaoyu Liu, Jordi Pons
We study permutation invariant training (PIT), which targets at the permutation ambiguity problem for speaker independent source separation models. We extend two state-of-the-art P…
Automatic multitrack mixing with a differentiable mixing console of neural audio effects
Christian J. Steinmetz, Jordi Pons, Santiago Pascual +1
Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relat…
Upsampling artifacts in neural audio synthesis
Jordi Pons, Santiago Pascual, Giulio Cengarle +1
A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied…
FSD50K: An Open Dataset of Human-Labeled Sound Events
Eduardo Fonseca, Xavier Favory, Jordi Pons +2
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2M tracks from YouTube videos an…
SESQA: semi-supervised learning for speech quality assessment
Joan Serrà, Jordi Pons, Santiago Pascual
Automatic speech quality assessment is an important, transversal task whose progress is hampered by the scarcity of human annotations, poor generalization to unseen recording condi…
TensorFlow Audio Models in Essentia
Pablo Alonso-Jiménez, Dmitry Bogdanov, Jordi Pons +1
Essentia is a reference open-source C++/Python library for audio and music analysis. In this work, we present a set of algorithms that employ TensorFlow in Essentia, allow predicti…
An empirical study of Conv-TasNet
Berkan Kadioglu, Michael Horgan, Xiaoyu Liu +3
Conv-TasNet is a recently proposed waveform-based deep neural network that achieves state-of-the-art performance in speech source separation. Its architecture consists of a learnab…
musicnn: Pre-trained convolutional neural networks for music audio tagging
Jordi Pons, Xavier Serra
Pronounced as "musician", the musicnn library contains a set of pre-trained musically motivated convolutional neural networks for music audio tagging: https://github.com/jordipons/…
Training neural audio classifiers with few data
Jordi Pons, Joan Serrà, Xavier Serra
We investigate supervised learning strategies that improve the training of neural network audio classifiers on small annotated collections. In particular, we study whether (i) a na…
End-to-end music source separation: is it possible in the waveform domain?
Francesc Lluís, Jordi Pons, Xavier Serra
Most of the currently successful source separation techniques use the magnitude spectrogram as input, and are therefore by default omitting part of the signal: the phase. To avoid…