Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory

Abstract

In regular statistical models, the leave-one-out cross-validation is asymptotically equivalent to the Akaike information criterion. However, since many learning machines are singular statistical models, the asymptotic behavior of the cross-validation remains unknown. In previous studies, we established the singular learning theory and proposed a widely applicable information criterion, the expectation value of which is asymptotically equal to the average Bayes generalization loss. In the present paper, we theoretically compare the Bayes cross-validation loss and the widely applicable information criterion and prove two theorems. First, the Bayes cross-validation loss is asymptotically equivalent to the widely applicable information criterion as a random variable. Therefore, model selection and hyperparameter optimization using these two values are asymptotically equivalent. Second, the sum of the Bayes generalization error and the Bayes cross-validation error is asymptotically equal to $2\lambda /n$, where $\lambda$ is the real log canonical threshold and $n$ is the number of training samples. Therefore the relation between the cross-validation error and the generalization error is determined by the algebraic geometrical structure of a learning machine. We also clarify that the deviance information criteria are different from the Bayes cross-validation and the widely applicable information criterion.

References in corpus (1)

• On real log canonical thresholds

Cited by in corpus (109)

• The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. II. UV, Optical, and Near-IR Light Curves and Comparison to Kilonova Models
• A Widely Applicable Bayesian Information Criterion
• The magnetar model for Type I superluminous supernovae I: Bayesian analysis of the full multicolour light curve sample with MOSFiT
• True Few-Shot Learning with Language Models
• Choice modelling in the age of machine learning -- discussion paper
• A thousand days after the merger: continued X-ray emission from GW170817
• Efficient Bayesian Structural Equation Modeling in Stan
• A Bayesian Cognition Approach to Improve Data Visualization
• Automatic cross-validation in structured models: Is it time to leave out leave-one-out?
• A resonant sextuplet of sub-Neptunes transiting the bright star HD 110067
• An extremely energetic supernova from a very massive star in a dense medium
• Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
• Unbiased and Efficient Log-Likelihood Estimation with Inverse Binomial Sampling
• A tale of two mergers: constraints on kilonova detection in two short GRBs at z$\sim$0.5
• Informed attribution of flood changes to decadal variation of atmospheric, catchment and river drivers in Upper Austria
• Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data
• Applying Bayesian Analysis Guidelines to Empirical Software Engineering Data: The Case of Programming Languages and Code Quality
• The role of alcohol outlet visits derived from mobile phone location data in enhancing domestic violence prediction at the neighborhood level
• Guidelines for the use of spatially-varying coefficients in species distribution models
• spAbundance: An R package for single-species and multi-species spatially explicit abundance models
• Robust leave-one-out cross-validation for high-dimensional Bayesian models
• Spatial Power Spectra of Dust across the Local Group: No Constraint on Disc Scale Height
• Latent Gaussian modeling and INLA: A review with focus on space-time applications
• Multivariate hierarchical analysis of car crashes data considering a spatial network lattice
• Big problems in spatio-temporal disease mapping: methods and software
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• PriorVAE: Encoding spatial priors with VAEs for small-area estimation
• Bayesian Non-Parametric Detection Heterogeneity in Ecological Models
• Scalable Bayesian Approach for the DINA Q-matrix Estimation Combining Stochastic Optimization and Variational Inference
• Bayesian Modeling of Dynamic Behavioral Change During an Epidemic
• Improving parametric mass modelling of lensing clusters through a perturbative approach
• Enhanced monitoring of atmospheric methane from space over the Permian basin with hierarchical Bayesian inference
• Heterogeneous Distributed Lag Models to Estimate Personalized Effects of Maternal Exposures to Air Pollution
• Fast and accurate Bayesian model criticism and conflict diagnostics using R-INLA
• High-dimensional order-free multivariate spatial disease mapping
• Modeling Extremal Streamflow using Deep Learning Approximations and a Flexible Spatial Process
• Integrated Nested Laplace Approximations (INLA)
• Bayesian Model Selection via Mean-Field Variational Approximation
• A Note on Bayesian Modeling Specification of Censored Data in JAGS
• Dimension-Grouped Mixed Membership Models for Multivariate Categorical Data
• A Bayesian Approach to Modelling Fine-Scale Spatial Dynamics of Non-State Terrorism: World Study, 2002-2013
• Bayesian Information Criterion for Linear Mixed-effects Models
• Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery
• A Bayesian approach to uncover local and temporal determinants of heterogeneity in repeated cross-sectional health surveys
• Bayesian Modal Regression based on Mixture Distributions
• Bayesian item response models for citizen science ecological data
• Income, education, and other poverty-related variables: a journey through Bayesian hierarchical models
• T-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs
• varstan: An R package for Bayesian analysis of structured time series models with Stan
• How to select an objective function using information theory
• Probabilistically-autoencoded horseshoe-disentangled multidomain item-response theory models
• Measurement Error Models for Spatial Network Lattice Data: Analysis of Car Crashes in Leeds
• What are the most important statistical ideas of the past 50 years?
• Warped Dynamic Linear Models for Time Series of Counts
• Bayesian Cross Validation and WAIC for Predictive Prior Design in Regular Asymptotic Theory
• Bayesian inference for dynamic vine copulas in higher dimensions
• Bigeminal Priors Variational auto-encoder
• A scalable approach for short-term disease forecasting in high spatial resolution areal data
• Personalized Dynamic Treatment Regimes in Continuous Time: A Bayesian Approach for Optimizing Clinical Decisions with Timing
• Stochastic Volatily Models using Hamiltonian Monte Carlo Methods and Stan
• Accounting for spatial varying sampling effort due to accessibility in Citizen Science data: A case study of moose in Norway
• A Novel Class of Unfolding Models for Binary Preference Data
• Adaptive preferential sampling in phylodynamics
• Divergent Effects of Factors on Crashes under Autonomous and Conventional Driving Modes Using A Hierarchical Bayesian Approach
• A Gentle Introduction to Bayesian Hierarchical Linear Regression Models
• Bayesian hierarchical space-time models to improve multispecies assessment by combining observations from disparate fish surveys
• Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization
• On prior smoothing with discrete spatial data in the context of disease mapping
• Bayesian Mendelian Randomization
• Estimating Heterogeneous Exposure Effects in the Case-Crossover Design using BART
• A priori truncation method for posterior sampling from homogeneous normalized completely random measure mixture models
• Approximate Bayesian Inference via Sparse grid Quadrature Evaluation for Hierarchical Models
• A Two-Stage Cox Process Model with Spatial and Nonspatial Covariates
• Simultaneous estimation of the effective reproducing number and the detection rate of COVID-19
• Bayesian Measurement Error Correction in Structured Additive Distributional Regression with an Application to the Analysis of Sensor Data on Soil-Plant Variability
• Large Deviation Strategy for Inverse Problem
• Non-Separable Spatio-temporal Models via Transformed Gaussian Markov Random Fields
• A Latent Space Model for Multilayer Network Data
• Hierarchical Multivariate Directed Acyclic Graph Auto-Regressive (MDAGAR) models for spatial diseases mapping
• Adaptive shrinkage of smooth functional effects towards a predefined functional subspace
• Empirical Bayes factors for common hypothesis tests
• A scale of interpretation for likelihood ratios and Bayes factors
• Prediction can be safely used as a proxy for explanation in causally consistent Bayesian generalized linear models
• Regularized Bayesian calibration and scoring of the WD-FAB IRT model improves predictive performance over marginal maximum likelihood
• The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation
• Modeling and presentation of vaccination coverage estimates using data from household surveys
• Bayesian Hierarchical Bernoulli-Weibull Mixture Model for Extremely Rare Events
• Bayesian Regression Markets
• Bivariate Analysis of Birth Weight and Gestational Age Depending on Environmental Exposures: Bayesian Distributional Regression with Copulas
• An Embedded Diachronic Sense Change Model with a Case Study from Ancient Greek
• CALF-SBM: A Covariate-Assisted Latent Factor Stochastic Block Model
• Flexible Validity Conditions for the Multivariate Matérn Covariance in any Spatial Dimension and for any Number of Components
• Addressing outliers in mixed-effects logistic regression: a more robust modeling approach
• Significant cocoon emission and photosphere duration stretching in GRB 211211A: a burst from a neutron star - black hole merger
• Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models
• Spatial modelling of COVID-19 incident cases using Richards' curve: an application to the Italian regions
• Bayesian model selection in the $\mathcal{M}$-open setting -- Approximate posterior inference and probability-proportional-to-size subsampling for efficient large-scale leave-one-out cross-validation
• A numerically stable online implementation and exploration of WAIC through variations of the predictive density, using NIMBLE
• Analysis of Prescription Drug Utilization with Beta Regression Models
• Bayesian modelling of statistical region- and family-level clustered ordinal outcome data from Turkey
• Testing for Typicality with Respect to an Ensemble of Learned Distributions
• Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion
• Joint Quantile Regression for Spatial Data
• The Quantum Relative Entropy as a Rate Function and Information Criteria
• A spatio-temporal model to understand forest fires causality in Europe
• Entropic Issues in Likelihood-Based OOD Detection
• Semiparametric Functional Factor Models with Bayesian Rank Selection
• Prior Intensified Information Criterion
• Dynamic Factor Models for Binary Data in Circular Spaces: An Application to the U.S. Supreme Court