237 citations · 1.1k across the 32 of their papers we have counts for
43 papers
CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Hao-Wen Dong, Xiaoyu Liu, Jordi Pons +5
Recent work has studied text-to-audio synthesis using large amounts of paired text-audio data. However, audio recordings with high-quality text annotations can be difficult to acqu…
Mono-to-stereo through parametric stereo generation
Joan Serrà, Davide Scaini, Santiago Pascual +4
Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific pa…
Full-band General Audio Synthesis with Score-based Diffusion
Santiago Pascual, Gautam Bhattacharya, Chunghsin Yeh +2
Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental s…
Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separation
Xiaoyu Liu, Xu Li, Joan Serrà
Single channel target speaker separation (TSS) aims at extracting a speaker's voice from a mixture of multiple talkers given an enrollment utterance of that speaker. A typical deep…
Adversarial Permutation Invariant Training for Universal Sound Separation
Emilian Postolache, Jordi Pons, Santiago Pascual +1
Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models t…
Universal Speech Enhancement with Score-based Diffusion
Joan Serrà, Santiago Pascual, Jordi Pons +2
Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings.…
On loss functions and evaluation metrics for music source separation
Enric Gusó, Jordi Pons, Santiago Pascual +1
We investigate which loss functions provide better separations via benchmarking an extensive set of those for music source separation. To that end, we first survey the most represe…
Upsampling layers for music source separation
Jordi Pons, Joan Serrà, Santiago Pascual +3
Upsampling artifacts are caused by problematic upsampling layers and due to spectral replicas that emerge while upsampling. Also, depending on the used upsampling layer, such artif…
Lognormals, Power Laws and Double Power Laws in the Distribution of Frequencies of Harmonic Codewords from Classical Music
Marc Serra-Peralta, Joan Serrà, Álvaro Corral
Zipf's law is a paradigm describing the importance of different elements in communication systems, especially in linguistics. Despite the complexity of the hierarchical structure o…
Assessing Algorithmic Biases for Musical Version Identification
Furkan Yesiler, Marius Miron, Joan Serrà +1
Version identification (VI) systems now offer accurate and scalable solutions for detecting different renditions of a musical composition, allowing the use of these systems in indu…
Audio-based Musical Version Identification: Elements and Challenges
Furkan Yesiler, Guillaume Doras, Rachel M. Bittner +2
In this article, we aim to provide a review of the key ideas and approaches proposed in 20 years of scientific literature around musical version identification (VI) research and co…
Adversarial Auto-Encoding for Packet Loss Concealment
Santiago Pascual, Joan Serrà, Jordi Pons
Communication technologies like voice over IP operate under constrained real-time conditions, with voice packets being subject to delays and losses from the network. In such cases,…
Heaps' Law and Vocabulary Richness in the History of Classical Music Harmony
Marc Serra-Peralta, Joan Serrà, Álvaro Corral
Music is a fundamental human construct, and harmony provides the building blocks of musical language. Using the Kunstderfuge corpus of classical music, we analyze the historical ev…
On tuning consistent annealed sampling for denoising score matching
Joan Serrà, Santiago Pascual, Jordi Pons
Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized…
Investigating the efficacy of music version retrieval systems for setlist identification
Furkan Yesiler, Emilio Molina, Joan Serrà +1
The setlist identification (SLI) task addresses a music recognition use case where the goal is to retrieve the metadata and timestamps for all the tracks played in live music event…
Automatic multitrack mixing with a differentiable mixing console of neural audio effects
Christian J. Steinmetz, Jordi Pons, Santiago Pascual +1
Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relat…
Less is more: Faster and better music version identification with embedding distillation
Furkan Yesiler, Joan Serrà, Emilia Gómez
Version identification systems aim to detect different renditions of the same underlying musical composition (loosely called cover songs). By learning to encode entire recordings i…
Upsampling artifacts in neural audio synthesis
Jordi Pons, Santiago Pascual, Giulio Cengarle +1
A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied…
SESQA: semi-supervised learning for speech quality assessment
Joan Serrà, Jordi Pons, Santiago Pascual
Automatic speech quality assessment is an important, transversal task whose progress is hampered by the scarcity of human annotations, poor generalization to unseen recording condi…
Accurate and Scalable Version Identification Using Musically-Motivated Embeddings
Furkan Yesiler, Joan Serrà, Emilia Gómez
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an ope…
Input complexity and out-of-distribution detection with likelihood-based generative models
Joan Serrà, David Álvarez, Vicenç Gómez +3
Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning sy…
Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion
Joan Serrà, Santiago Pascual, Carlos Segura
End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion,…
Towards Generalized Speech Enhancement with Generative Adversarial Networks
Santiago Pascual, Joan Serrà, Antonio Bonafonte
The speech enhancement task usually consists of removing additive noise or reverberation that partially mask spoken utterances, affecting their intelligibility. However, little att…
Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks
Santiago Pascual, Mirco Ravanelli, Joan Serrà +2
Learning good representations without supervision is still an open issue in machine learning, and is particularly challenging for speech signals, which are often characterized by l…