116 citations · 349 across the 21 of their papers we have counts for
24 papers · 1 filter
GASS: Generalizing Audio Source Separation with Large-scale Data
Jordi Pons, Xiaoyu Liu, Santiago Pascual +1
Universal source separation targets at separating the audio sources of an arbitrary mix, removing the constraint to operate on a specific domain like speech or music. Yet, the pote…
CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Hao-Wen Dong, Xiaoyu Liu, Jordi Pons +5
Recent work has studied text-to-audio synthesis using large amounts of paired text-audio data. However, audio recordings with high-quality text annotations can be difficult to acqu…
Mono-to-stereo through parametric stereo generation
Joan Serrà, Davide Scaini, Santiago Pascual +4
Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific pa…
Full-band General Audio Synthesis with Score-based Diffusion
Santiago Pascual, Gautam Bhattacharya, Chunghsin Yeh +2
Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental s…
Adversarial Permutation Invariant Training for Universal Sound Separation
Emilian Postolache, Jordi Pons, Santiago Pascual +1
Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models t…
PodcastMix: A dataset for separating music and speech in podcasts
Nicolás Schmidt, Jordi Pons, Marius Miron
We introduce PodcastMix, a dataset formalizing the task of separating background music and foreground speech in podcasts. We aim at defining a benchmark suitable for training and e…
Universal Speech Enhancement with Score-based Diffusion
Joan Serrà, Santiago Pascual, Jordi Pons +2
Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings.…
On loss functions and evaluation metrics for music source separation
Enric Gusó, Jordi Pons, Santiago Pascual +1
We investigate which loss functions provide better separations via benchmarking an extensive set of those for music source separation. To that end, we first survey the most represe…
Upsampling layers for music source separation
Jordi Pons, Joan Serrà, Santiago Pascual +3
Upsampling artifacts are caused by problematic upsampling layers and due to spectral replicas that emerge while upsampling. Also, depending on the used upsampling layer, such artif…
Adversarial Auto-Encoding for Packet Loss Concealment
Santiago Pascual, Joan Serrà, Jordi Pons
Communication technologies like voice over IP operate under constrained real-time conditions, with voice packets being subject to delays and losses from the network. In such cases,…
Multichannel-based learning for audio object extraction
Daniel Arteaga, Jordi Pons
The current paradigm for creating and deploying immersive audio content is based on audio objects, which are composed of an audio track and position metadata. While rendering an ob…
On permutation invariant training for speech source separation
Xiaoyu Liu, Jordi Pons
We study permutation invariant training (PIT), which targets at the permutation ambiguity problem for speaker independent source separation models. We extend two state-of-the-art P…
Upsampling artifacts in neural audio synthesis
Jordi Pons, Santiago Pascual, Giulio Cengarle +1
A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied…
FSD50K: An Open Dataset of Human-Labeled Sound Events
Eduardo Fonseca, Xavier Favory, Jordi Pons +2
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2M tracks from YouTube videos an…
musicnn: Pre-trained convolutional neural networks for music audio tagging
Jordi Pons, Xavier Serra
Pronounced as "musician", the musicnn library contains a set of pre-trained musically motivated convolutional neural networks for music audio tagging: https://github.com/jordipons/…
Training neural audio classifiers with few data
Jordi Pons, Joan Serrà, Xavier Serra
We investigate supervised learning strategies that improve the training of neural network audio classifiers on small annotated collections. In particular, we study whether (i) a na…
End-to-end music source separation: is it possible in the waveform domain?
Francesc Lluís, Jordi Pons, Xavier Serra
Most of the currently successful source separation techniques use the magnitude spectrogram as input, and are therefore by default omitting part of the signal: the phase. To avoid…
General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline
Eduardo Fonseca, Manoj Plakal, Frederic Font +4
This paper describes Task 2 of the DCASE 2018 Challenge, titled "General-purpose audio tagging of Freesound content with AudioSet labels". This task was hosted on the Kaggle platfo…
Randomly weighted CNNs for (music) audio classification
Jordi Pons, Xavier Serra
The computer vision literature shows that randomly weighted neural networks perform reasonably as feature extractors. Following this idea, we study how non-trained (randomly weight…
End-to-end learning for music audio tagging at scale
Jordi Pons, Oriol Nieto, Matthew Prockup +3
The lack of data tends to limit the outcomes of deep learning research, particularly when dealing with end-to-end learning stacks processing raw data such as waveforms. In this stu…
Audio to score matching by combining phonetic and duration information
Rong Gong, Jordi Pons, Xavier Serra
We approach the singing phrase audio to score matching problem by using phonetic and duration information - with a focus on studying the jingju a cappella singing case. We argue th…
Score-informed syllable segmentation for a cappella singing voice with convolutional neural networks
Jordi Pons, Rong Gong, Xavier Serra
This paper introduces a new score-informed method for the segmentation of jingju a cappella singing phrase into syllables. The proposed method estimates the most likely sequence of…
A Wavenet for Speech Denoising
Dario Rethage, Jordi Pons, Xavier Serra
Currently, most speech processing techniques use magnitude spectrograms as front-end and are therefore by default discarding part of the signal: the phase. In order to overcome thi…
Timbre Analysis of Music Audio Signals with Convolutional Neural Networks
Jordi Pons, Olga Slizovskaia, Rong Gong +2
The focus of this work is to study how to efficiently tailor Convolutional Neural Networks (CNNs) towards learning timbre representations from log-mel magnitude spectrograms. We fi…