A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms

Abstract

The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed "ForneyLab" (https://github.com/biaslab/ForneyLab.jl) as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools "Edward" and "Stan". In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.

arXiv commentAccepted for publication in the International Journal of Approximate Reasoning

References in corpus (10)

• Variational Inference: A Review for Statisticians
• Expectation Propagation for approximate Bayesian inference
• Edward: A library for probabilistic modeling, inference, and criticism
• REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
• Automatic Variational Inference in Stan
• ZhuSuan: A Library for Bayesian Deep Learning
• Accelerating Inference: towards a full Language, Compiler and Hardware stack
• Expectation Particle Belief Propagation
• Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
• Bayes Blocks: An Implementation of the Variational Bayesian Building Blocks Framework

Cited by in corpus (8)

• Julia Language in Machine Learning: Algorithms, Applications, and Open Issues
• Application of the Free Energy Principle to Estimation and Control
• Active Inference and Epistemic Value in Graphical Models
• Chance-Constrained Active Inference
• AIDA: An Active Inference-based Design Agent for Audio Processing Algorithms
• Realising Synthetic Active Inference Agents, Part II: Variational Message Updates
• pymdp: A Python library for active inference in discrete state spaces
• Automating Model Comparison in Factor Graphs