30 citations · 149 across the 35 of their papers we have counts for
45 papers
TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description
Mingyang Yao, Jingyue Huang, Ke Chen +2
Text-conditioned symbolic music generation has attracted growing interest with the development of generative models. Yet two questions remain underexplored: what aspects of music t…
Relational Synthesis: Structure-Mediated Concatenative Synthesis for Foley and Retrieval-Augmented Audio Generation
Keren Shao, Ayaka Kawano, Shlomo Dubnov
We ask: given a retrieved source audio and a separate reference audio , can we synthesize novel audio out of this pair such that remains acoustically consist…
Action-Directed Information for Distributed Control and Agentic Interaction
Shlomo Dubnov
Distributed intelligence concerns systems in which semi-autonomous components with local dynamics and partial observations coordinate through information exchange to maintain a sha…
Towards Real-Time Human-AI Musical Co-Performance: Accompaniment Generation with Latent Diffusion Models and MAX/MSP
Tornike Karchkhadze, Shlomo Dubnov
We present a framework for real-time human-AI musical co-performance, in which a latent diffusion model generates instrumental accompaniment in response to a live stream of context…
BACHI: Boundary-Aware Symbolic Chord Recognition Through Masked Iterative Decoding on Pop and Classical Music
Mingyang Yao, Ke Chen, Shlomo Dubnov +1
Automatic chord recognition (ACR) via deep learning models has gradually achieved promising recognition accuracy, yet two key challenges remain. First, prior work has primarily foc…
kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimization
Keren Shao, Ke Chen, Matthew Baas +1
Robustness is critical in zero-shot singing voice conversion (SVC). This paper introduces two novel methods to strengthen the robustness of the kNN-VC framework for SVC. First, kNN…
Synthesizing Composite Hierarchical Structure from Symbolic Music Corpora
Ilana Shapiro, Ruanqianqian Huang, Zachary Novack +5
Western music is an innately hierarchical system of interacting levels of structure, from fine-grained melody to high-level form. In order to analyze music compositions holisticall…
Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
Tornike Karchkhadze, Keren Shao, Shlomo Dubnov
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By levera…
Improving Music Source Separation with Diffusion and Consistency Refinement
Tornike Karchkhadze, Mohammad Rasool Izadi, Shuo Zhang +1
In this work, we propose an approach to music source separation that uses a generative diffusion model as a last-stage refinement on top of a deterministic separator, progressively…
Creativity and Visual Communication from Machine to Musician: Sharing a Score through a Robotic Camera
Ross Greer, Laura Fleig, Shlomo Dubnov
This paper explores the integration of visual communication and musical interaction by implementing a robotic camera within a "Guided Harmony" musical game. We aim to examine co-cr…
Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model
Tornike Karchkhadze, Mohammad Rasool Izadi, Ke Chen +2
Diffusion models have shown promising results in cross-modal generation tasks involving audio and music, such as text-to-sound and text-to-music generation. These text-controlled m…
Generating Symbolic Music from Natural Language Prompts using an LLM-Enhanced Dataset
Weihan Xu, Julian McAuley, Taylor Berg-Kirkpatrick +2
Recent years have seen many audio-domain text-to-music generation models that rely on large amounts of text-audio pairs for training. However, symbolic-domain controllable music ge…
Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models
Tornike Karchkhadze, Mohammad Rasool Izadi, Shlomo Dubnov
Diffusion models have recently shown strong potential in both music generation and music source separation tasks. Although in early stages, a trend is emerging towards integrating…
Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation
Ke Chen, Jiaqi Su, Taylor Berg-Kirkpatrick +2
Achieving robust speech separation for overlapping speakers in various acoustic environments with noise and reverberation remains an open challenge. Although existing datasets are…
Music Enhancement with Deep Filters: A Technical Report for The ICASSP 2024 Cadenza Challenge
Keren Shao, Ke Chen, Shlomo Dubnov
In this challenge, we disentangle the deep filters from the original DeepfilterNet and incorporate them into our Spec-UNet-based network to further improve a hybrid Demucs (hdemucs…
Evaluating Co-Creativity using Total Information Flow
Vignesh Gokul, Chris Francis, Shlomo Dubnov
Co-creativity in music refers to two or more musicians or musical agents interacting with one another by composing or improvising music. However, this is a very subjective process…
Binaural sound source localization using a hybrid time and frequency domain model
Gil Geva, Olivier Warusfel, Shlomo Dubnov +3
This paper introduces a new approach to sound source localization using head-related transfer function (HRTF) characteristics, which enable precise full-sphere localization from ra…
PosCUDA: Position based Convolution for Unlearnable Audio Datasets
Vignesh Gokul, Shlomo Dubnov
Deep learning models require large amounts of clean data to acheive good performance. To avoid the cost of expensive data acquisition, researchers use the abundant data available o…
Equipping Pretrained Unconditional Music Transformers with Instrument and Genre Controls
Weihan Xu, Julian McAuley, Shlomo Dubnov +1
The ''pretraining-and-finetuning'' paradigm has become a norm for training domain-specific models in natural language processing and computer vision. In this work, we aim to examin…
SelfVC: Voice Conversion With Iterative Refinement using Self Transformations
Paarth Neekhara, Shehzeen Hussain, Rafael Valle +5
We propose SelfVC, a training strategy to iteratively improve a voice conversion model with self-synthesized examples. Previous efforts on voice conversion focus on factorizing spe…
Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction
Keren Shao, Ke Chen, Taylor Berg-Kirkpatrick +1
In deep learning research, many melody extraction models rely on redesigning neural network architectures to improve performance. In this paper, we propose an input feature modific…
MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
Ke Chen, Yusong Wu, Haohe Liu +3
Diffusion models have shown promising results in cross-modal generation tasks, including text-to-image and text-to-audio generation. However, generating music, as a special type of…
Universal Source Separation with Weakly Labelled Data
Qiuqiang Kong, Ke Chen, Haohe Liu +4
Universal source separation (USS) is a fundamental research task for computational auditory scene analysis, which aims to separate mono recordings into individual source tracks. Th…
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
Yusong Wu, Ke Chen, Tianyu Zhang +4
Contrastive learning has shown remarkable success in the field of multimodal representation learning. In this paper, we propose a pipeline of contrastive language-audio pretraining…