24 citations · 97 across the 31 of their papers we have counts for
35 papers
TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description
Mingyang Yao, Jingyue Huang, Ke Chen +2
Text-conditioned symbolic music generation has attracted growing interest with the development of generative models. Yet two questions remain underexplored: what aspects of music t…
Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning
Hansen Jin Lillemark, Alex Rojas, Zachary Novack +5
Standard autoregressive video generation algorithms based on Diffusion and Flow Matching rely on rigid training objectives and static sampling schedules, limiting inference procedu…
Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators
Zachary Novack, Stephen Brade, Haven Kim +8
Interactive streaming music generation promises the use of generative models for live performance and co-creation that is impossible with offline models. However, SOTA models exist…
MuseTok: Symbolic Music Tokenization for Generation and Semantic Understanding
Jingyue Huang, Zachary Novack, Phillip Long +4
Discrete representation learning has shown promising results across various domains, including generation and understanding in image, speech and language. Inspired by these advance…
Steering Autoregressive Music Generation with Recursive Feature Machines
Daniel Zhao, Daniel Beaglehole, Taylor Berg-Kirkpatrick +2
Controllable music generation remains a significant challenge, with existing methods often requiring model retraining or introducing audible artifacts. We introduce MusicRFM, a fra…
BACHI: Boundary-Aware Symbolic Chord Recognition Through Masked Iterative Decoding on Pop and Classical Music
Mingyang Yao, Ke Chen, Shlomo Dubnov +1
Automatic chord recognition (ACR) via deep learning models has gradually achieved promising recognition accuracy, yet two key challenges remain. First, prior work has primarily foc…
WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling
Qihui Yang, Taylor Berg-Kirkpatrick, Julian McAuley +1
Despite rapid progress in end-to-end AI music generation, AI-driven modeling of professional Digital Signal Processing (DSP) workflows remains challenging. In particular, while the…
REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing
Weihan Xu, Yimeng Ma, Jingyue Huang +6
Short videos are an effective tool for promoting contents and improving knowledge accessibility. While existing extractive video summarization methods struggle to produce a coheren…
Fast Text-to-Audio Generation with Adversarial Post-Training
Zachary Novack, Zach Evans, Zack Zukowski +8
Text-to-audio systems, while increasingly performant, are slow at inference time, thus making their latency unpractical for many creative applications. We present Adversarial Relat…
Are you really listening? Boosting Perceptual Awareness in Music-QA Benchmarks
Yongyi Zang, Sean O'Brien, Taylor Berg-Kirkpatrick +2
Large Audio Language Models (LALMs), where pretrained text LLMs are finetuned with audio input, have made remarkable progress in music understanding. However, current evaluation me…
Synthesizing Composite Hierarchical Structure from Symbolic Music Corpora
Ilana Shapiro, Ruanqianqian Huang, Zachary Novack +5
Western music is an innately hierarchical system of interacting levels of structure, from fine-grained melody to high-level form. In order to analyze music compositions holisticall…
TeaserGen: Generating Teasers for Long Documentaries
Weihan Xu, Paul Pu Liang, Haven Kim +3
Teasers are an effective tool for promoting content in entertainment, commercial and educational fields. However, creating an effective teaser for long videos is challenging for it…
Presto! Distilling Steps and Layers for Accelerating Music Generation
Zachary Novack, Ge Zhu, Jonah Casebeer +3
Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration…
Generating Symbolic Music from Natural Language Prompts using an LLM-Enhanced Dataset
Weihan Xu, Julian McAuley, Taylor Berg-Kirkpatrick +2
Recent years have seen many audio-domain text-to-music generation models that rely on large amounts of text-audio pairs for training. However, symbolic-domain controllable music ge…
PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processing
Phillip Long, Zachary Novack, Taylor Berg-Kirkpatrick +1
The recent explosion of generative AI-Music systems has raised numerous concerns over data copyright, licensing music from musicians, and the conflict between open-source AI and la…
Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation
Ke Chen, Jiaqi Su, Taylor Berg-Kirkpatrick +2
Achieving robust speech separation for overlapping speakers in various acoustic environments with noise and reverberation remains an open challenge. Although existing datasets are…
DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick +1
Controllable music generation methods are critical for human-centered AI-based music creation, but are currently limited by speed, quality, and control design trade-offs. Diffusion…
DITTO: Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick +1
We propose Diffusion Inference-Time T-Optimization (DITTO), a general-purpose frame-work for controlling pre-trained text-to-music diffusion models at inference-time via optimizing…
Unsupervised Lead Sheet Generation via Semantic Compression
Zachary Novack, Nikita Srivatsan, Taylor Berg-Kirkpatrick +1
Lead sheets have become commonplace in generative music research, being used as an initial compressed representation for downstream tasks like multitrack music generation and autom…
Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction
Keren Shao, Ke Chen, Taylor Berg-Kirkpatrick +1
In deep learning research, many melody extraction models rely on redesigning neural network architectures to improve performance. In this paper, we propose an input feature modific…
MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
Ke Chen, Yusong Wu, Haohe Liu +3
Diffusion models have shown promising results in cross-modal generation tasks, including text-to-image and text-to-audio generation. However, generating music, as a special type of…
CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Hao-Wen Dong, Xiaoyu Liu, Jordi Pons +5
Recent work has studied text-to-audio synthesis using large amounts of paired text-audio data. However, audio recordings with high-quality text annotations can be difficult to acqu…
Universal Source Separation with Weakly Labelled Data
Qiuqiang Kong, Ke Chen, Haohe Liu +4
Universal source separation (USS) is a fundamental research task for computational auditory scene analysis, which aims to separate mono recordings into individual source tracks. Th…
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji +2
Recent years have seen progress beyond domain-specific sound separation for speech or music towards universal sound separation for arbitrary sounds. Prior work on universal sound s…