Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

Abstract

The idea of computer vision as the Bayesian inverse problem to computer graphics has a long history and an appealing elegance, but it has proved difficult to directly implement. Instead, most vision tasks are approached via complex bottom-up processing pipelines. Here we show that it is possible to write short, simple probabilistic graphics programs that define flexible generative models and to automatically invert them to interpret real-world images. Generative probabilistic graphics programs consist of a stochastic scene generator, a renderer based on graphics software, a stochastic likelihood model linking the renderer's output and the data, and latent variables that adjust the fidelity of the renderer and the tolerance of the likelihood model. Representations and algorithms from computer graphics, originally designed to produce high-quality images, are instead used as the deterministic backbone for highly approximate and stochastic generative models. This formulation combines probabilistic programming, computer graphics, and approximate Bayesian computation, and depends only on general-purpose, automatic inference techniques. We describe two applications: reading sequences of degraded and adversarially obscured alphanumeric characters, and inferring 3D road models from vehicle-mounted camera images. Each of the probabilistic graphics programs we present relies on under 20 lines of probabilistic code, and supports accurate, approximately Bayesian inferences about ambiguous real-world images.

The first two authors contributed equally to this work

References in corpus (1)

• Church: a language for generative models

Cited by in corpus (19)

• Deep Convolutional Inverse Graphics Network
• Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
• Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars
• Scaling Exact Inference for Discrete Probabilistic Programs
• Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• On the Binding Problem in Artificial Neural Networks
• The Informed Sampler: A Discriminative Approach to Bayesian Inference in Generative Computer Vision Models
• Neural Approximate Sufficient Statistics for Implicit Models
• Designing Perceptual Puzzles by Differentiating Probabilistic Programs
• ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
• Neural Density Estimation and Likelihood-free Inference
• Disentangled Representations in Neural Models
• Transflow Learning: Repurposing Flow Models Without Retraining
• Probably approximate Bayesian computation: nonasymptotic convergence of ABC under misspecification
• 3DP3: 3D Scene Perception via Probabilistic Programming
• Neural Articulated Radiance Field
• Analytical Derivatives for Differentiable Renderer: 3D Pose Estimation by Silhouette Consistency
• Real-time Approximate Bayesian Computation for Scene Understanding