Structured Output Learning with Conditional Generative Flows
arXiv:1905.13288
Abstract
Traditional structured prediction models try to learn the conditional likelihood, i.e., p(y|x), to capture the relationship between the structured output y and the input features x. For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objectives or variational inference to approximate likelihood. In this paper, we propose conditional Glow (c-Glow), a conditional generative flow for structured output learning. C-Glow benefits from the ability of flow-based models to compute p(y|x) exactly and efficiently. Learning with c-Glow does not require a surrogate objective or performing inference during training. Once trained, we can directly and efficiently generate conditional samples. We develop a sample-based prediction method, which can use this advantage to do efficient and effective inference. In our experiments, we test c-Glow on five different tasks. C-Glow outperforms the state-of-the-art baselines in some tasks and predicts comparable outputs in the other tasks. The results show that c-Glow is versatile and is applicable to many different structured prediction problems.
arXiv commentAccepted to AAAI 2020
References in corpus (18)
- Adam: A Method for Stochastic Optimization
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- Conditional Random Fields as Recurrent Neural Networks
- NICE: Non-linear Independent Components Estimation
- Density estimation using Real NVP
- A note on the evaluation of generative models
- Generative Image Inpainting with Contextual Attention
- Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Learning Deep Structured Models
- Latent Normalizing Flows for Discrete Sequences
- Learning Approximate Inference Networks for Structured Prediction
- Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
- Deep Learning for Semantic Part Segmentation with High-Level Guidance
- Conditional Density Estimation with Bayesian Normalising Flows
- Deep Structured Prediction with Nonlinear Output Transformations
- MaCow: Masked Convolutional Generative Flow