OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression

Abstract

Explicit deep generative models (DGMs), e.g., VAEs and Normalizing Flows, have shown to offer an effective data modelling alternative for lossless compression. However, DGMs themselves normally require large storage space and thus contaminate the advantage brought by accurate data density estimation. To eliminate the requirement of saving separate models for different target datasets, we propose a novel setting that starts from a pretrained deep generative model and compresses the data batches while adapting the model with a dynamical system for only one epoch. We formalise this setting as that of One-Shot Online Adaptation (OSOA) of DGMs for lossless compression and propose a vanilla algorithm under this setting. Experimental results show that vanilla OSOA can save significant time versus training bespoke models and space versus using one model for all targets. With the same adaptation step number or adaptation time, it is shown vanilla OSOA can exhibit better space efficiency, e.g., $47\%$ less space, than fine-tuning the pretrained model and saving the fine-tuned model. Moreover, we showcase the potential of OSOA and motivate more sophisticated OSOA algorithms by showing further space or time efficiency with multiple updates per batch and early stopping.

References in corpus (14)

• Auto-Encoding Variational Bayes
• YFCC100M: The New Data in Multimedia Research
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Video Compression With Rate-Distortion Autoencoders
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding
• RNADE: The real-valued neural autoregressive density-estimator
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• The Description Length of Deep Learning Models
• Compression with Flows via Local Bits-Back Coding
• VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
• Overfitting for Fun and Profit: Instance-Adaptive Data Compression
• Learning Discrete Distributions by Dequantization