1.4k citations · 1.9k across the 13 of their papers we have counts for
18 papers
Sample what you cant compress
Vighnesh Birodkar, Gabriel Barcik, James Lyon +3
For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial…
VideoPoet: A Large Language Model for Zero-Shot Video Generation
Dan Kondratyuk, Lijun Yu, Xiuye Gu +28
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals. VideoPoet employs a decoder-on…
Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression
David Minnen, Nick Johnston
The rate-distortion performance of neural image compression models has exceeded the state-of-the-art for non-learned codecs, but neural codecs are still far from widespread deploym…
Finite Scalar Quantization: VQ-VAE Made Simple
Fabian Mentzer, David Minnen, Eirikur Agustsson +1
We propose to replace vector quantization (VQ) in the latent representation of VQ-VAEs with a simple scheme termed finite scalar quantization (FSQ), where we project the VAE repres…
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
Lijun Yu, José Lezama, Nitesh B. Gundavarapu +13
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation. To effec…
Multi-Realism Image Compression with a Conditional Generator
Eirikur Agustsson, David Minnen, George Toderici +1
By optimizing the rate-distortion-realism trade-off, generative compression approaches produce detailed, realistic images, even at low bit rates, instead of the blurry reconstructi…
VCT: A Video Compression Transformer
Fabian Mentzer, George Toderici, David Minnen +4
We show how transformers can be used to vastly simplify neural video compression. Previous methods have been relying on an increasing number of architectural biases and priors, inc…
Neural Video Compression using GANs for Detail Synthesis and Propagation
Fabian Mentzer, Eirikur Agustsson, Johannes Ballé +3
We present the first neural video compression method based on generative adversarial networks (GANs). Our approach significantly outperforms previous neural and non-neural video co…
Channel-wise Autoregressive Entropy Models for Learned Image Compression
David Minnen, Saurabh Singh
In learning-based approaches to image compression, codecs are developed by optimizing a computational model to minimize a rate-distortion objective. Currently, the most effective l…
Nonlinear Transform Coding
Johannes Ballé, Philip A. Chou, David Minnen +5
We review a class of methods that can be collected under the name nonlinear transform coding (NTC), which over the past few years have become competitive with the best linear trans…
Joint Autoregressive and Hierarchical Priors for Learned Image Compression
David Minnen, Johannes Ballé, George Toderici
Recent models for learned image compression are based on autoencoders, learning approximately invertible mappings from pixels to a quantized latent representation. These are combin…
Towards a Semantic Perceptual Image Metric
Troy Chinen, Johannes Ballé, Chunhui Gu +8
We present a full reference, perceptual image metric based on VGG-16, an artificial neural network trained on object classification. We fit the metric to a new database based on 14…
Image-Dependent Local Entropy Models for Learned Image Compression
David Minnen, George Toderici, Saurabh Singh +2
The leading approach for image compression with artificial neural networks (ANNs) is to learn a nonlinear transform and a fixed entropy model that are optimized for rate-distortion…
Spatially adaptive image compression using a tiled deep network
David Minnen, George Toderici, Michele Covell +6
Deep neural networks represent a powerful class of function approximators that can learn to compress and reconstruct images. Existing image compression algorithms based on neural n…
Variational image compression with a scale hyperprior
Johannes Ballé, David Minnen, Saurabh Singh +2
We describe an end-to-end trainable model for image compression based on variational autoencoders. The model incorporates a hyperprior to effectively capture spatial dependencies i…
Target-Quality Image Compression with Recurrent, Convolutional Neural Networks
Michele Covell, Nick Johnston, David Minnen +5
We introduce a stop-code tolerant (SCT) approach to training recurrent convolutional neural networks for lossy image compression. Our methods introduce a multi-pass training method…
Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks
Nick Johnston, Damien Vincent, David Minnen +6
We propose a method for lossy image compression based on recurrent, convolutional neural networks that outperforms BPG (4:2:0 ), WebP, JPEG2000, and JPEG as measured by MS-SSIM. We…
Full Resolution Image Compression with Recurrent Neural Networks
George Toderici, Damien Vincent, Nick Johnston +4
This paper presents a set of full-resolution lossy image compression methods based on neural networks. Each of the architectures we describe can provide variable compression rates…