2.3k citations · 2.8k across the 17 of their papers we have counts for
19 papers
Anticipatory Music Transformer
John Thickstun, David Hall, Chris Donahue +1
We introduce anticipation: a method for constructing a controllable generative model of a temporal point process (the event process) conditioned asynchronously on realizations of a…
V2Meow: Meowing to the Visual Beat via Video-to-Music Generation
Kun Su, Judith Yue Li, Qingqing Huang +8
Video-to-music generation demands both a temporally localized high-quality listening experience and globally aligned video-acoustic signatures. While recent music generation models…
SingSong: Generating musical accompaniments from singing
Chris Donahue, Antoine Caillon, Adam Roberts +8
We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create mus…
Melody transcription via generative pre-training
Chris Donahue, John Thickstun, Percy Liang
Despite the central role that melody plays in music perception, it remains an open challenge in music information retrieval to reliably detect the notes of the melody present in an…
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Abstract semantic 3D scene understanding is a problem of critical importance in robotics. As robots still lack the common-sense knowledge about household objects and locations of a…
It's Raw! Audio Generation with State-Space Models
Karan Goel, Albert Gu, Chris Donahue +1
Developing architectures suitable for modeling raw audio is a challenging problem due to the high sampling rates of audio waveforms. Standard sequence modeling approaches like RNNs…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
Codified audio language modeling learns useful representations for music information retrieval
Rodrigo Castellon, Chris Donahue, Percy Liang
We demonstrate that language models pre-trained on codified (discretely-encoded) music audio learn representations that are useful for downstream MIR tasks. Specifically, we explor…
Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music
Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick +1
Modern keyboards allow a musician to play multiple instruments at the same time by assigning zones -- fixed pitch ranges of the keyboard -- to different instruments. In this paper,…
Enabling Language Models to Fill in the Blanks
Chris Donahue, Mina Lee, Percy Liang
We present a simple approach for text infilling, the task of predicting missing spans of text at any position in a document. While infilling could enable rich functionality especia…
LakhNES: Improving multi-instrumental music generation with cross-domain pre-training
Chris Donahue, Huanru Henry Mao, Yiting Ethan Li +2
We are interested in the task of generating multi-instrumental music scores. The Transformer architecture has recently shown great promise for the task of piano score generation; h…
Expediting TTS Synthesis with Adversarial Vocoding
Paarth Neekhara, Chris Donahue, Miller Puckette +2
Recent approaches in text-to-speech (TTS) synthesis employ neural network strategies to vocode perceptually-informed spectrogram representations directly into listenable waveforms.…
GANSynth: Adversarial Neural Audio Synthesis
Jesse Engel, Kumar Krishna Agrawal, Shuo Chen +3
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregress…
Piano Genie
Chris Donahue, Ian Simon, Sander Dieleman
We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons,…
The NES Music Database: A multi-instrumental dataset with expressive performance attributes
Chris Donahue, Huanru Henry Mao, Julian McAuley
Existing research on music generation focuses on composition, but often ignores the expressive performance characteristics required for plausible renditions of resultant pieces. In…
Adversarial Audio Synthesis
Chris Donahue, Julian McAuley, Miller Puckette
Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales. Generative adversarial networks…
Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition
Chris Donahue, Bo Li, Rohit Prabhavalkar
We investigate the effectiveness of generative adversarial networks (GANs) for speech enhancement, in the context of improving noise robustness of automatic speech recognition (ASR…
Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
Chris Donahue, Zachary C. Lipton, Akshay Balsubramani +1
We propose a new algorithm for training generative adversarial networks that jointly learns latent codes for both identities (e.g. individual humans) and observations (e.g. specifi…
Dance Dance Convolution
Chris Donahue, Zachary C. Lipton, Julian McAuley
Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players perform steps on a dance platform in synchronization with music as directed by on-screen step charts. Whi…