3 citations · 4 across the 11 of their papers we have counts for
10 papers · 1 filter
A Generative-First Neural Audio Autoencoder
Jonah Casebeer, Ge Zhu, Zhepei Wang +1
Neural autoencoders underpin generative models. Practical, large-scale use of neural autoencoders for generative modeling necessitates fast encoding, low latent rates, and a single…
TAC: Timestamped Audio Captioning
Sonal Kumar, Prem Seetharaman, Ke Chen +8
Large Audio Language Models struggle to disentangle overlapping events in complex acoustic scenes, yielding temporally inconsistent captions and frequent hallucinations. We introdu…
Stemphonic: All-at-once Flexible Multi-stem Music Generation
Shih-Lun Wu, Ge Zhu, Juan-Pablo Caceres +2
Music stem generation, the task of producing musically-synchronized and isolated instrument audio clips, offers the potential of greater user control and better alignment with musi…
Rethinking Music Captioning with Music Metadata LLMs
Irmak Bukey, Zhepei Wang, Chris Donahue +1
Music captioning, or the task of generating a natural language description of music, is useful for both music understanding and controllable music generation. Training captioning m…
DRAGON: Distributional Rewards Optimize Diffusion Generative Models
Yatong Bai, Jonah Casebeer, Somayeh Sojoudi +1
We present Distributional RewArds for Generative OptimizatioN (DRAGON), a versatile framework for fine-tuning media generation models towards a desired outcome. Compared with tradi…
Presto! Distilling Steps and Layers for Accelerating Music Generation
Zachary Novack, Ge Zhu, Jonah Casebeer +3
Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration…
DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick +1
Controllable music generation methods are critical for human-centered AI-based music creation, but are currently limited by speed, quality, and control design trade-offs. Diffusion…
MusicHiFi: Fast High-Fidelity Stereo Vocoding
Ge Zhu, Juan-Pablo Caceres, Zhiyao Duan +1
Diffusion-based audio and music generation models commonly perform generation by constructing an image representation of audio (e.g., a mel-spectrogram) and then convert it to audi…
DITTO: Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick +1
We propose Diffusion Inference-Time T-Optimization (DITTO), a general-purpose frame-work for controlling pre-trained text-to-music diffusion models at inference-time via optimizing…
Music ControlNet: Multiple Time-varying Controls for Music Generation
Shih-Lun Wu, Chris Donahue, Shinji Watanabe +1
Text-to-music generation models are now capable of generating high-quality music audio in broad styles. However, text control is primarily suitable for the manipulation of global m…