Evaluating structure learning algorithms with a balanced scoring function

Abstract

Several structure learning algorithms have been proposed towards discovering causal or Bayesian Network (BN) graphs. The validity of these algorithms tends to be evaluated by assessing the relationship between the learnt and the ground truth graph. However, there is no agreed scoring metric to determine this relationship. Moreover, this paper shows that some of the commonly used metrics tend to be biased in favour of graphs that minimise edges. While graphs that are less complex are desirable, some of the metrics favour underfitted graphs, thereby encouraging limited propagation of evidence. This paper proposes the Balanced Scoring Function (BSF) that eliminates this bias by adjusting the reward function based on the difficulty of discovering an edge, or no edge, proportional to their occurrence rate in the ground truth graph. The BSF score can be used in conjunction with other traditional metrics to provide an alternative and unbiased assessment about the capability of a structure learning algorithm in discovering causal or BN graphs.

References in corpus (19)

• Causal Inference and Causal Explanation with Background Knowledge
• Estimating high-dimensional intervention effects from observational data
• A Transformational Characterization of Equivalent Bayesian Network Structures
• Causal Inference in the Presence of Latent Variables and Selection Bias
• Bayesian network learning with cutting planes
• DAG-GNN: DAG Structure Learning with Graph Neural Networks
• Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs
• A Discovery Algorithm for Directed Cyclic Graphs
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Bayesian structure learning using dynamic programming and MCMC
• Learning Bayesian Networks from Incomplete Data with Stochastic Search Algorithms
• Bayesian network learning by compiling to weighted MAX-SAT
• Learning Structures of Bayesian Networks for Variable Groups
• SparsityBoost: A New Scoring Function for Learning Bayesian Network Structure
• Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures
• Bayesian Network Structure Learning with Permutation Tests
• Scoring and Searching over Bayesian Networks with Causal and Associative Priors
• Robust learning Bayesian networks for prior belief
• An Algorithm for Learning the Essential Graph

Cited by in corpus (2)

• Bayesian network structure learning with causal effects in the presence of latent variables
• Effective and efficient structure learning with pruning and model averaging strategies