Model selection and local geometry
arXiv:1801.08364 · doi:10.1214/19-AOS1940
Abstract
We consider problems in model selection caused by the geometry of models close to their points of intersection. In some cases---including common classes of causal or graphical models, as well as time series models---distinct models may nevertheless have identical tangent spaces. This has two immediate consequences: first, in order to obtain constant power to reject one model in favour of another we need local alternative hypotheses that decrease to the null at a slower rate than the usual parametric (typically we will require or slower); in other words, to distinguish between the models we need large effect sizes or very large sample sizes. Second, we show that under even weaker conditions on their tangent cones, models in these classes cannot be made simultaneously convex by a reparameterization. This shows that Bayesian network models, amongst others, cannot be learned directly with a convex method similar to the graphical lasso. However, we are able to use our results to suggest methods for model selection that learn the tangent space directly, rather than the model itself. In particular, we give a generic algorithm for learning Bayesian network models.
References in corpus (13)
- High-dimensional graphs and variable selection with the Lasso
- Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness and sensitivity analysis
- CAM: Causal additive models, high-dimensional order search and penalized regression
- Geometry of the faithfulness assumption in causal inference
- Likelihood ratio tests and singularities
- Discrete chain graph models
- Nested Markov Properties for Acyclic Directed Mixed Graphs
- Concave Penalized Estimation of Sparse Gaussian Bayesian Networks
- The log-linear group-lasso estimator and its asymptotic properties
- Penalized Estimation of Directed Acyclic Graphs From Discrete Data
- Asymptotic distribution and sparsistency for l1-penalized parametric M-estimators with applications to linear SVM and logistic regression
- Smoothness of marginal log-linear parameterizations
- Sparse Nested Markov models with Log-linear Parameters