3.6k citations · 6.2k across the 27 of their papers we have counts for
30 papers
Introduction to Stochastic Differential Equations for Generative Machine Learning: A Variational Perspective
Ole Winther, Paul Jeha, Sander Dieleman +3
The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomole…
An Efficient Algorithm for Thresholding Monte Carlo Tree Search
Shoma Nameki, Atsuyoshi Nakamura, Junpei Komiyama +1
We introduce the Thresholding Monte Carlo Tree Search problem, in which, given a tree and a threshold , a player must answer whether the root node value of $\mathc…
The NeurIPS 2023 Machine Learning for Audio Workshop: Affective Audio Benchmarks and Novel Data
Alice Baird, Rachel Manzelli, Panagiotis Tzirakis +7
The NeurIPS 2023 Machine Learning for Audio Workshop brings together machine learning (ML) experts from various audio domains. There are several valuable audio-driven ML tasks, fro…
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
Yilun Du, Conor Durkan, Robin Strudel +6
Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of…
Continuous diffusion for categorical data
Sander Dieleman, Laurent Sartran, Arman Roshannai +11
Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges…
Self-conditioned Embedding Diffusion for Text Generation
Robin Strudel, Corentin Tallec, Florent Altché +8
Can continuous diffusion models bring the same performance breakthrough on natural language they did for image generation? To circumvent the discrete nature of text data, we can si…
Categorical SDEs with Simplex Diffusion
Pierre H. Richemond, Sander Dieleman, Arnaud Doucet
Diffusion models typically operate in the standard framework of generative modelling by producing continuously-valued datapoints. To this end, they rely on a progressive Gaussian s…
General-purpose, long-context autoregressive modeling with Perceiver AR
Curtis Hawthorne, Andrew Jaegle, Cătălina Cangea +12
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression. However, the most commonly u…
Towards Learning Universal Audio Representations
Luyu Wang, Pauline Luc, Yan Wu +8
The ability to learn universal audio representations that can solve diverse speech, music, and environment tasks can spur many applications that require general sound content under…
Variable-rate discrete representation learning
Sander Dieleman, Charlie Nash, Jesse Engel +1
Semantically meaningful information content in perceptual signals is usually unevenly distributed. In speech signals for example, there are often many silences, and the speed of pr…
Generating Images with Sparse Representations
Charlie Nash, Jacob Menick, Sander Dieleman +1
The high dimensionality of images presents architecture and sampling-efficiency challenges for likelihood-based generative models. Previous approaches such as VQ-VAE use deep autoe…
A Deep Learning Approach for Characterizing Major Galaxy Mergers
Skanda Koppula, Victor Bapst, Marc Huertas-Company +15
Fine-grained estimation of galaxy merger stages from observations is a key problem useful for validation of our current theoretical understanding of galaxy formation. To this end,…
Towards transformation-resilient provenance detection of digital media
Jamie Hayes, Krishnamurthy, Dvijotham +4
Advancements in deep generative models have made it possible to synthesize images, videos and audio signals that are difficult to distinguish from natural signals, creating opportu…
Self-Supervised MultiModal Versatile Networks
Jean-Baptiste Alayrac, Adrià Recasens, Rosalia Schneider +6
Videos are a rich source of multi-modal supervision. In this work, we learn representations using self-supervision by leveraging three modalities naturally present in videos: visua…
End-to-End Adversarial Text-to-Speech
Jeff Donahue, Sander Dieleman, Mikołaj Bińkowski +2
Modern text-to-speech synthesis pipelines typically involve multiple processing stages, each of which is designed or learnt independently from the rest. In this work, we take on th…
Transformation-based Adversarial Video Prediction on Large-Scale Data
Pauline Luc, Aidan Clark, Sander Dieleman +4
Recent breakthroughs in adversarial generative modeling have led to models capable of producing video samples of high quality, even on large and complex datasets of real-world vide…
High Fidelity Speech Synthesis with Adversarial Networks
Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman +5
Generative adversarial networks have seen rapid development in recent years and have led to remarkable improvements in generative modelling of images. However, their application in…
Hierarchical Autoregressive Image Models with Auxiliary Decoders
Jeffrey De Fauw, Sander Dieleman, Karen Simonyan
Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale…
Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts +6
Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales. Fortunately, most mu…
Piano Genie
Chris Donahue, Ian Simon, Sander Dieleman
We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons,…
This Time with Feeling: Learning Expressive Musical Performance
Sageev Oore, Ian Simon, Sander Dieleman +2
Music generation has generally been focused on either creating scores or interpreting them. We discuss differences between these two problems and propose that, in fact, it may be v…
The challenge of realistic music generation: modelling raw audio at scale
Sander Dieleman, Aäron van den Oord, Karen Simonyan
Realistic music generation is a challenging task. When building generative models of music that are learnt from data, typically high-level representations such as scores or MIDI ar…
Efficient Neural Audio Synthesis
Nal Kalchbrenner, Erich Elsen, Karen Simonyan +7
Sequential models achieve state-of-the-art results in audio, visual and textual domains with respect to both estimating the data distribution and generating high-quality samples. E…
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Aaron van den Oord, Yazhe Li, Igor Babuschkin +19
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages…