Nonparametric Conditional Density Estimation in a High-Dimensional Regression Setting

Abstract

In some applications (e.g., in cosmology and economics), the regression E[Z|x] is not adequate to represent the association between a predictor x and a response Z because of multi-modality and asymmetry of f(z|x); using the full density instead of a single-point estimate can then lead to less bias in subsequent analysis. As of now, there are no effective ways of estimating f(z|x) when x represents high-dimensional, complex data. In this paper, we propose a new nonparametric estimator of f(z|x) that adapts to sparse (low-dimensional) structure in x. By directly expanding f(z|x) in the eigenfunctions of a kernel-based operator, we avoid tensor products in high dimensions as well as ratios of estimated densities. Our basis functions are orthogonal with respect to the underlying data distribution, allowing fast implementation and tuning of parameters. We derive rates of convergence and show that the method adapts to the intrinsic dimension of the data. We also demonstrate the effectiveness of the series method on images, spectra, and an application to photometric redshift estimation of galaxies.

References in corpus (2)

• What Lies Beneath: Using p(z) to Reduce Systematic Photometric Redshift Errors
• Randomized partition trees for exact nearest neighbor search

Cited by in corpus (15)

• Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference
• Spotting Anomalous Trades in NFT Markets: The Case of NBA Topshot
• A Unified Framework for Constructing, Tuning and Assessing Photometric Redshift Density Estimates in a Selection Bias Setting
• Flexible distribution-free conditional predictive bands using density estimators
• CD-split and HPD-split: efficient conformal regions in high dimensions
• Bayesian Non-parametric Quantile Process Regression and Estimation of Marginal Quantile Effects
• Conformal Prediction using Conditional Histograms
• The Quasar Catalogue for S-PLUS DR4 (QuCatS) and the estimation of photometric redshifts
• Improved Weak Lensing Photometric Redshift Calibration via StratLearn and Hierarchical Modeling
• Deep Distribution Regression
• A Tale of Two Bases: Local-Nonlocal Regularization on Image Patches with Convolution Framelets
• Adaptive greedy algorithm for moderately large dimensions in kernel conditional density estimation
• Dirac Delta Regression: Conditional Density Estimation with Clinical Trials
• Quantifying Epistemic Uncertainty in Deep Learning
• A Deep Generative Approach to Conditional Sampling