Fast Nonparametric Conditional Density Estimation

Abstract

Conditional density estimation generalizes regression by modeling a full density f(yjx) rather than only the expected value E(yjx). This is important for many tasks, including handling multi-modality and generating prediction intervals. Though fundamental and widely applicable, nonparametric conditional density estimators have received relatively little attention from statisticians and little or none from the machine learning community. None of that work has been applied to greater than bivariate data, presumably due to the computational difficulty of data-driven bandwidth selection. We describe the double kernel conditional density estimator and derive fast dual-tree-based algorithms for bandwidth selection using a maximum likelihood criterion. These techniques give speedups of up to 3.8 million in our experiments, and enable the first applications to previously intractable large multivariate datasets, including a redshift prediction problem from the Sloan Digital Sky Survey.

Appears in Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence (UAI2007)

Cited by in corpus (11)

• Scenario Parameter Generation Method and Scenario Representativeness Metric for Scenario-Based Assessment of Automated Vehicles
• PRISMA: A Novel Approach for Deriving Probabilistic Surrogate Safety Measures for Risk Evaluation
• Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification
• Bayesian Non-parametric Quantile Process Regression and Estimation of Marginal Quantile Effects
• Constrained Sampling from a Kernel Density Estimator to Generate Scenarios for the Assessment of Automated Vehicles
• Bottleneck Conditional Density Estimation
• Probabilistic supervised learning
• Conditional Density Estimation, Latent Variable Discovery and Optimal Transport
• Minimax Rates for Conditional Density Estimation via Empirical Entropy
• Learning Transfer Operators by Kernel Density Estimation
• Deep Distribution Regression