7 papers
V2M-Zero: Zero-Pair Time-Aligned Video-to-Music Generation
Yan-Bo Lin, Jonah Casebeer, Long Mai +3
Generating music that temporally aligns with video events is challenging for existing text-to-music models, which lack fine-grained temporal control. We introduce V2M-ZERO, a video…
A Generative-First Neural Audio Autoencoder
Jonah Casebeer, Ge Zhu, Zhepei Wang +1
Neural autoencoders underpin generative models. Practical, large-scale use of neural autoencoders for generative modeling necessitates fast encoding, low latent rates, and a single…
DRAGON: Distributional Rewards Optimize Diffusion Generative Models
Yatong Bai, Jonah Casebeer, Somayeh Sojoudi +1
We present Distributional RewArds for Generative OptimizatioN (DRAGON), a versatile framework for fine-tuning media generation models towards a desired outcome. Compared with tradi…
Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders
Dimitrios Bralios, Jonah Casebeer, Paris Smaragdis
Neural audio codecs and autoencoders have emerged as versatile models for audio compression, transmission, feature-extraction, and latent-space generation. However, a key limitatio…
Learning to Upsample and Upmix Audio in the Latent Domain
Dimitrios Bralios, Paris Smaragdis, Jonah Casebeer
Neural audio autoencoders create compact latent representations that preserve perceptually important information, serving as the foundation for both modern audio compression system…
Presto! Distilling Steps and Layers for Accelerating Music Generation
Zachary Novack, Ge Zhu, Jonah Casebeer +3
Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration…