Parameter Learning of Logic Programs for Symbolic-Statistical Modeling

Abstract

We propose a logical/mathematical framework for statistical parameter learning of parameterized logic programs, i.e. definite clause programs containing probabilistic facts with a parameterized distribution. It extends the traditional least Herbrand model semantics in logic programming to distribution semantics, possible world semantics with a probability distribution which is unconditionally applicable to arbitrary logic programs including ones for HMMs, PCFGs and Bayesian networks. We also propose a new EM algorithm, the graphical EM algorithm, that runs for a class of parameterized logic programs representing sequential decision processes where each decision is exclusive and independent. It runs on a new data structure called support graphs describing the logical relationship between observations and their explanations, and learns parameters by computing inside and outside probability generalized for logic programs. The complexity analysis shows that when combined with OLDT search for all explanations for observations, the graphical EM algorithm, despite its generality, has the same time complexity as existing EM algorithms, i.e. the Baum-Welch algorithm for HMMs, the Inside-Outside algorithm for PCFGs, and the one for singly connected Bayesian networks that have been developed independently in each research field. Learning experiments with PCFGs using two corpora of moderate size indicate that the graphical EM algorithm can significantly outperform the Inside-Outside algorithm.

References in corpus (2)

• Loglinear models for first-order probabilistic reasoning
• Probabilistic Deduction with Conditional Constraints over Basic Events

Cited by in corpus (27)

• On the Implementation of the Probabilistic Logic Programming Language ProbLog
• CLP(BN): Constraint Logic Programming for Probabilistic Knowledge
• Logical Hidden Markov Models
• Structure Learning of Probabilistic Logic Programs by Searching the Clause Space
• Location-Based Reasoning about Complex Multi-Agent Behavior
• The Magic of Logical Inference in Probabilistic Programming
• Inference in Probabilistic Logic Programs with Continuous Random Variables
• Nesting Probabilistic Inference
• Representing Conversations for Scalable Overhearing
• The Language Features and Architecture of B-Prolog
• Markov Logic Networks for Natural Language Question Answering
• Infinite probability computation by cyclic explanation graphs
• Viterbi training in PRISM
• Parameter Learning in PRISM Programs with Continuous Random Variables
• PASOCS: A Parallel Approximate Solver for Probabilistic Logic Programs under the Credal Semantics
• Speeding-up ProbLog's Parameter Learning
• DNF Sampling for ProbLog Inference
• MatSat: a matrix-based differentiable SAT solver
• A Logic-based Approach to Generatively Defined Discriminative Modeling
• Reasoning with Probabilistic Logics
• The Complexity of Bayesian Networks Specified by Propositional and Relational Languages
• Value of Information in Probabilistic Logic Programs
• Formal Verification using Second-Quantized Horn Clauses
• Probabilistic Logic Programming with Beta-Distributed Random Variables
• Optimizing Probabilities in Probabilistic Logic Programs
• Deriving a Stationary Dynamic Bayesian Network from a Logic Program with Recursive Loops
• On the Semantics and Complexity of Probabilistic Logic Programs