BART: Bayesian additive regression trees

Abstract

We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Published in at http://dx.doi.org/10.1214/09-AOAS285 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (90)

• Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Estimating treatment effect heterogeneity in randomized program evaluation
• Discussion on "Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects" by Hahn, Murray and Carvalho
• A Review of Generalizability and Transportability
• A Survey of Learning Causality with Data: Problems and Methods
• A review of predictive uncertainty estimation with machine learning
• Variable selection for BART: An application to gene regulation
• Data-Driven Modelling of the Reynolds Stress Tensor using Random Forests with Invariance
• Deep Learning: A Bayesian Perspective
• Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models
• Bayesian additive regression trees and the General BART model
• Assessing lack of common support in causal inference using Bayesian nonparametrics: Implications for evaluating the effect of breastfeeding on children's cognitive outcomes
• Estimating heterogeneous survival treatment effect in observational data using machine learning
• Augmenting Interpretable Models with LLMs during Training
• A Bayesian Additive Model for Understanding Public Transport Usage in Special Events
• Individualized Prediction of COVID-19 Adverse outcomes with MLHO
• Conjugate Bayes for probit regression via unified skew-normal distributions
• Minimax-optimal nonparametric regression in high dimensions
• Adaptive network reliability analysis: Methodology and applications to power grid
• A New Spatial Count Data Model with Bayesian Additive Regression Trees for Accident Hot Spot Identification
• Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression
• A Practical Introduction to Bayesian Estimation of Causal Effects: Parametric and Nonparametric Approaches
• Model interpretation through lower-dimensional posterior summarization
• Multiple imputation for sharing precise geographies in public use data
• Offline Contextual Multi-armed Bandits for Mobile Health Interventions: A Case Study on Emotion Regulation
• Variance prior forms for high-dimensional Bayesian variable selection
• Bayesian Additive Regression Trees with Model Trees
• The scatter in the galaxy-halo connection: a machine learning analysis
• Estimating Perinatal Critical Windows of Susceptibility to Environmental Mixtures via Structured Bayesian Regression Tree Pairs
• DESCN: Deep Entire Space Cross Networks for Individual Treatment Effect Estimation
• Machine Learning applied to asteroid dynamics
• Sequential Design for Ranking Response Surfaces
• Experimental Evaluation of Individualized Treatment Rules
• Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data
• Supervised learning for the prediction of firm dynamics
• A Bayesian Machine Learning Approach for Estimating Heterogeneous Survivor Causal Effects: Applications to a Critical Care Trial
• Variable selection and sensitivity analysis using dynamic trees, with an application to computer code performance tuning
• Estimating a Causal Exposure Response Function with a Continuous Error-Prone Exposure: A Study of Fine Particulate Matter and All-Cause Mortality
• Matching in Selective and Balanced Representation Space for Treatment Effects Estimation
• A Bayesian regression tree approach to identify the effect of nanoparticles' properties on toxicity profiles
• A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites
• Performance of variable and function selection methods for estimating the non-linear health effects of correlated chemical mixtures: a simulation study
• A Bayesian Nonparametric Model for Zero-Inflated Outcomes: Prediction, Clustering, and Causal Estimation
• A Decision-Theoretic Approach for Model Interpretability in Bayesian Framework
• Sequential Bayesian experimental design for calibration of expensive simulation models
• Pareto Invariant Representation Learning for Multimedia Recommendation
• Suicide disparities across urban and suburban areas in the U.S.: A comparative assessment of socio-environmental factors using a data-driven predictive approach
• Imputation procedures in surveys using nonparametric and machine learning methods: an empirical comparison
• Heterogeneous Distributed Lag Models to Estimate Personalized Effects of Maternal Exposures to Air Pollution
• Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
• Integrating Nearest Neighbors with Neural Network Models for Treatment Effect Estimation
• Bayesian Synthesis: Combining subjective analyses, with an application to ozone data
• Quantifying sources of uncertainty in drug discovery predictions with probabilistic models
• "Robust-squared" Imputation Models Using BART
• Interpolating between small- and large-$g$ expansions using Bayesian Model Mixing
• LogitBoost autoregressive networks
• Bayesian Semiparametric Model for Sequential Treatment Decisions with Informative Timing
• Fast Deep Mixtures of Gaussian Process Experts
• Variable Selection with ABC Bayesian Forests
• Accounting for selection bias due to death in estimating the effect of wealth shock on cognition for the Health and Retirement Study
• Robust discrete choice models with t-distributed kernel errors
• What Makes Forest-Based Heterogeneous Treatment Effect Estimators Work?
• Influencing elections with statistics: Targeting voters with logistic regression trees
• Efficient Bayesian Multivariate Surface Regression
• Bayesian non-parametric ordinal regression under a monotonicity constraint
• Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Population
• What is to be gained by ensemble models in analysis of spectroscopic data?
• A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting
• Assisting Clinical Decisions for Scarcely Available Treatment via Disentangled Latent Representation
• Neural Greedy Pursuit for Feature Selection
• Accounting for shared covariates in semi-parametric Bayesian additive regression trees
• A flexible Bayesian g-formula for causal survival analyses with time-dependent confounding
• Operator-induced structural variable selection for identifying materials genes
• Comparison of methods for analyzing environmental mixtures effects on survival outcomes and application to a population-based cohort study
• Predicting Exporters with Machine Learning
• Estimating Heterogeneous Exposure Effects in the Case-Crossover Design using BART
• The Taxicab Sampler: MCMC for Discrete Spaces with Application to Tree Models
• Evaluating plate discipline in Major League Baseball with Bayesian Additive Regression Trees
• New directions in algebraic statistics: Three challenges from 2023
• Walking fingerprinting
• Counterfactual Prediction Under Selective Confounding
• Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimation
• Mid infrared spectroscopy and milk quality traits: a data analysis competition at the "International Workshop on Spectroscopy and Chemometrics 2021"
• Horseshoe Forests for High-Dimensional Causal Survival Analysis
• Bayesian Semi-supervised Inference via a Debiased Modeling Approach
• Hierarchical Bayesian Bootstrap for Heterogeneous Treatment Effect Estimation
• Conformal prediction for uncertainties in nucleon-nucleon scattering
• Only-child matching penalty in the marriage market
• Quantum Speedups for Multiproposal MCMC