The Informed Sampler: A Discriminative Approach to Bayesian Inference in Generative Computer Vision Models

Abstract

Computer vision is hard because of a large variability in lighting, shape, and texture; in addition the image signal is non-additive due to occlusion. Generative models promised to account for this variability by accurately modelling the image formation process as a function of latent variables with prior beliefs. Bayesian posterior inference could then, in principle, explain the observation. While intuitively appealing, generative models for computer vision have largely failed to deliver on that promise due to the difficulty of posterior inference. As a result the community has favoured efficient discriminative approaches. We still believe in the usefulness of generative models in computer vision, but argue that we need to leverage existing discriminative or even heuristic computer vision methods. We implement this idea in a principled way with an "informed sampler" and in careful experiments demonstrate it on challenging generative models which contain renderer programs as their components. We concentrate on the problem of inverting an existing graphics rendering engine, an approach that can be understood as "Inverse Graphics". The informed sampler, using simple discriminative proposals based on existing computer vision technology, achieves significant improvements of inference.

Appearing in Computer Vision and Image Understanding Journal (Special Issue on Generative Models in Computer Vision)

References in corpus (3)

• MCMC using Hamiltonian dynamics
• Markov Chain Monte Carlo: Can We Trust the Third Significant Figure?
• Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

Cited by in corpus (12)

• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• Inference Networks for Sequential Monte Carlo in Graphical Models
• Towards causal generative scene models via competition of experts
• Learning to superoptimize programs
• Inverse Graphics with Probabilistic CAD Models
• Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation
• Informed MCMC with Bayesian Neural Networks for Facial Image Analysis
• Learning Direct Optimization for Scene Understanding
• Consensus Message Passing for Layered Graphical Models
• Applications of Probabilistic Programming (Master's thesis, 2015)
• Unsupervised Object Learning via Common Fate
• Inner Space Preserving Generative Pose Machine