Testing Goodness of Fit of Conditional Density Models with Kernels
arXiv:2002.10271
Abstract
We propose two nonparametric statistical tests of goodness of fit for conditional distributions: given a conditional probability density function and a joint sample, decide whether the sample is drawn from for some density . Our tests, formulated with a Stein operator, can be applied to any differentiable conditional density model, and require no knowledge of the normalizing constant. We show that 1) our tests are consistent against any fixed alternative conditional model; 2) the statistics can be estimated easily, requiring no density estimation as an intermediate step; and 3) our second test offers an interpretable test result providing insight on where the conditional model does not fit well in the domain of the covariate. We demonstrate the interpretability of our test on a task of modeling the distribution of New York City's taxi drop-off location given a pick-up point. To our knowledge, our work is the first to propose such conditional goodness-of-fit tests that simultaneously have all these desirable properties.
arXiv commentIn UAI 2020. http://auai.org/uai2020/accepted.php
References in corpus (13)
- Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
- A Kernel Test of Goodness of Fit
- Measuring Sample Quality with Kernels
- Fast Two-Sample Testing with Analytic Representations of Probability Measures
- A Linear-Time Kernel Goodness-of-Fit Test
- Characteristic and Universal Tensor Product Kernels
- A Test of Relative Similarity For Model Selection in Generative Models
- Gaussian Process Conditional Density Estimation
- Random Feature Stein Discrepancies
- An Adaptive Test of Independence with Analytic Kernel Embeddings
- Informative Features for Model Comparison
- Vector valued reproducing kernel Hilbert spaces and universality
- Comparing distributions: geometry improves kernel two-sample testing