InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers

Abstract

Continuous Normalizing Flows (CNFs) have emerged as promising deep generative models for a wide range of tasks thanks to their invertibility and exact likelihood estimation. However, conditioning CNFs on signals of interest for conditional image generation and downstream predictive tasks is inefficient due to the high-dimensional latent code generated by the model, which needs to be of the same size as the input data. In this paper, we propose InfoCNF, an efficient conditional CNF that partitions the latent space into a class-specific supervised code and an unsupervised code that shared among all classes for efficient use of labeled information. Since the partitioning strategy (slightly) increases the number of function evaluations (NFEs), InfoCNF also employs gating networks to learn the error tolerances of its ordinary differential equation (ODE) solvers for better speed and performance. We show empirically that InfoCNF improves the test accuracy over the baseline while yielding comparable likelihood scores and reducing the NFEs on CIFAR10. Furthermore, applying the same partitioning strategy in InfoCNF on time-series data helps improve extrapolation performance.

17 pages, 14 figures, 2 tables

References in corpus (10)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Variational Inference with Normalizing Flows
• Large Batch Training of Convolutional Networks
• Adaptive Computation Time for Recurrent Neural Networks
• Latent ODEs for Irregularly-Sampled Time Series
• Analyzing Inverse Problems with Invertible Neural Networks
• Hybrid Models with Deep and Invertible Features
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Conditional Adversarial Generative Flow for Controllable Image Synthesis

Cited by in corpus (6)

• Turbulence forecasting via Neural ODE
• ECINN: Efficient Counterfactuals from Invertible Neural Networks
• Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
• Neural ODEs for Image Segmentation with Level Sets
• Second-Order Neural ODE Optimizer
• Viscos Flows: Variational Schur Conditional Sampling With Normalizing Flows