Quasi-Bayesian Dual Instrumental Variable Regression

Abstract

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking. In this work we present a novel quasi-Bayesian procedure for IV regression, building upon the recently developed kernelized IV models and the dual/minimax formulation of IV regression. We analyze the frequentist behavior of the proposed method, by establishing minimax optimal contraction rates in $L_2$ and Sobolev norms, and discussing the frequentist validity of credible balls. We further derive a scalable inference algorithm which can be extended to work with wide neural network models. Empirical evaluation shows that our method produces informative uncertainty estimates on complex high-dimensional problems.

arXiv commentExtended version of the NeurIPS'21 paper. ZW and YZ contribute equally

References in corpus (17)

• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• An MCMC Approach to Classical Estimation
• Rates of contraction of posterior distributions based on Gaussian process priors
• Bayesian inverse problems with Gaussian priors
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Reproducing kernel Hilbert spaces of Gaussian priors
• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems
• Quasi-Bayesian analysis of nonparametric instrumental variables models
• Frequentist coverage and sup-norm convergence rate in Gaussian process regression
• Bayesian Deep Ensembles via the Neural Tangent Kernel
• Adversarial Generalized Method of Moments
• Dual Instrumental Variable Regression
• Instrumental Variable Regression via Kernel Maximum Moment Loss
• Analysis and Probability on Infinite-Dimensional Spaces
• Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction