9 papers
Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators
Zachary Novack, Stephen Brade, Haven Kim +8
Interactive streaming music generation promises the use of generative models for live performance and co-creation that is impossible with offline models. However, SOTA models exist…
You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
Zhepei Wei, Xinyu Zhu, Wei-Lin Chen +3
Reinforcement learning with verifiable rewards (RLVR) has become a dominant paradigm for improving reasoning in large language models (LLMs), yet the underlying geometry of the res…
Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction
Yusong Wu, Stephen Brade, Aleksandra Teng Ma +6
Most applications of generative AI involve a sequential interaction in which a person inputs a prompt and waits for a response, and where reaction time and adaptivity are not impor…
Streaming Generation for Music Accompaniment
Yusong Wu, Mason Wang, Heidi Lei +5
Music generation models can produce high-fidelity coherent accompaniment given complete audio input, but are limited to editing and loop-based workflows. We study real-time audio-t…
Adaptive Accompaniment with ReaLchords
Yusong Wu, Tim Cooijmans, Kyle Kastner +10
Jamming requires coordination, anticipation, and collaborative creativity between musicians. Current generative models of music produce expressive output but are not able to genera…
FLAM: Frame-Wise Language-Audio Modeling
Yusong Wu, Christos Tsirigotis, Ke Chen +5
Recent multi-modal audio-language models (ALMs) excel at text-audio retrieval but struggle with frame-wise audio understanding. Prior works use temporal-aware labels or unsupervise…
ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers
Alexander Scarlatos, Yusong Wu, Ian Simon +5
Recent advances in generative artificial intelligence (AI) have created models capable of high-quality musical content generation. However, little consideration is given to how to…
Exploratory Study Of Human-AI Interaction For Hindustani Music
Nithya Shikarpur, Cheng-Zhi Anna Huang
This paper presents a study of participants interacting with and using GaMaDHaNi, a novel hierarchical generative model for Hindustani vocal contours. To explore possible use cases…
Hierarchical Generative Modeling of Melodic Vocal Contours in Hindustani Classical Music
Nithya Shikarpur, Krishna Maneesha Dendukuri, Yusong Wu +2
Hindustani music is a performance-driven oral tradition that exhibits the rendition of rich melodic patterns. In this paper, we focus on generative modeling of singers' vocal melod…