2k citations · 4.6k across the 27 of their papers we have counts for
29 papers
Live Music Models
Lyria Team, Antoine Caillon, Brian McWilliams +33
We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release M…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
Michaël Mathieu, Sherjil Ozair, Srivatsan Srinivasan +21
StarCraft II is one of the most challenging simulated reinforcement learning environments; it is partially observable, stochastic, multi-agent, and mastering StarCraft II requires…
Step-unrolled Denoising Autoencoders for Text Generation
Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski +2
In this paper we propose a new generative model of text, Step-unrolled Denoising Autoencoder (SUNDAE), that does not rely on autoregressive models. Similarly to denoising diffusion…
Towards Learning Universal Audio Representations
Luyu Wang, Pauline Luc, Yan Wu +8
The ability to learn universal audio representations that can solve diverse speech, music, and environment tasks can spur many applications that require general sound content under…
Vector Quantized Models for Planning
Sherjil Ozair, Yazhe Li, Ali Razavi +3
Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been…
Divide and Contrast: Self-supervised Learning from Uncurated Data
Yonglong Tian, Olivier J. Henaff, Aaron van den Oord
Self-supervised learning holds promise in leveraging large amounts of unlabeled data, however much of its progress has thus far been limited to highly curated pre-training data suc…
Multimodal Self-Supervised Learning of General Audio Representations
Luyu Wang, Pauline Luc, Adria Recasens +2
We present a multimodal framework to learn general audio representations from videos. Existing contrastive audio representation learning methods mainly focus on using the audio mod…
Multi-Format Contrastive Learning of Audio Representations
Luyu Wang, Aaron van den Oord
Recent advances suggest the advantage of multi-modal training in comparison with single-modal methods. In contrast to this view, in our work we find that similar gain can be obtain…
Predicting Video with VQVAE
Jacob Walker, Ali Razavi, Aäron van den Oord
In recent years, the task of video prediction-forecasting future video given past video frames-has attracted attention in the research community. In this paper we propose a novel a…
Broaden Your Views for Self-Supervised Video Learning
Adrià Recasens, Pauline Luc, Jean-Baptiste Alayrac +11
Most successful self-supervised learning methods are trained to align the representations of two independent views from the data. State-of-the-art methods in video are inspired by…
Efficient Visual Pretraining with Contrastive Detection
Olivier J. Hénaff, Skanda Koppula, Jean-Baptiste Alayrac +3
Self-supervised pretraining has been shown to yield powerful representations for transfer learning. These performance gains come at a large computational cost however, with state-o…
Are we done with ImageNet?
Lucas Beyer, Olivier J. Hénaff, Alexander Kolesnikov +2
Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overf…
Learning Robust and Multilingual Speech Representations
Kazuya Kawakami, Luyu Wang, Chris Dyer +2
Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognitio…
Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
Cristina Gârbacea, Aäron van den Oord, Yazhe Li +4
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with…
Shaping Belief States with Generative Environment Models for RL
Karol Gregor, Danilo Jimenez Rezende, Frederic Besse +3
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressiv…
On Variational Bounds of Mutual Information
Ben Poole, Sherjil Ozair, Aaron van den Oord +2
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and s…
Data-Efficient Image Recognition with Contrastive Predictive Coding
Olivier J. Hénaff, Aravind Srinivas, Jeffrey De Fauw +4
Human observers can learn to recognize new categories of images from a handful of examples, yet doing so with artificial ones remains an open challenge. We hypothesize that data-ef…
Wasserstein Dependency Measure for Representation Learning
Sherjil Ozair, Corey Lynch, Yoshua Bengio +3
Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as…
Preventing Posterior Collapse with delta-VAEs
Ali Razavi, Aäron van den Oord, Ben Poole +1
Due to the phenomenon of "posterior collapse," current latent variable generative models pose a challenging design choice that either weakens the capacity of the decoder or require…
Sample Efficient Adaptive Text-to-Speech
Yutian Chen, Yannis Assael, Brendan Shillingford +11
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and ind…
The challenge of realistic music generation: modelling raw audio at scale
Sander Dieleman, Aäron van den Oord, Karen Simonyan
Realistic music generation is a challenging task. When building generative models of music that are learnt from data, typically high-level representations such as scores or MIDI ar…
Associative Compression Networks for Representation Learning
Alex Graves, Jacob Menick, Aaron van den Oord
This paper introduces Associative Compression Networks (ACNs), a new framework for variational autoencoding with neural networks. The system differs from existing variational autoe…
Efficient Neural Audio Synthesis
Nal Kalchbrenner, Erich Elsen, Karen Simonyan +7
Sequential models achieve state-of-the-art results in audio, visual and textual domains with respect to both estimating the data distribution and generating high-quality samples. E…