Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings

Abstract

Current meta-learning approaches focus on learning functional representations of relationships between variables, i.e. on estimating conditional expectations in regression. In many applications, however, we are faced with conditional distributions which cannot be meaningfully summarized using expectation only (due to e.g. multimodality). Hence, we consider the problem of conditional density estimation in the meta-learning setting. We introduce a novel technique for meta-learning which combines neural representation and noise-contrastive estimation with the established literature of conditional mean embeddings into reproducing kernel Hilbert spaces. The method is validated on synthetic and real-world problems, demonstrating the utility of sharing learned representations across multiple conditional density estimation tasks.

References in corpus (13)

• Representation Learning with Contrastive Predictive Coding
• A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
• Learning Deep Kernels for Non-Parametric Two-Sample Tests
• Conditional Neural Processes
• Conditional Density Estimation with Bayesian Normalising Flows
• Gaussian Process Conditional Density Estimation
• Attentive Neural Processes
• A Comparative Analysis of Expected and Distributional Reinforcement Learning
• Kernel Exponential Family Estimation via Doubly Dual Embedding
• Learning deep kernels for exponential family densities
• Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency
• Conditional mean embeddings as regressors - supplementary
• Kernel Conditional Exponential Family

Cited by in corpus (3)

• Function Contrastive Learning of Transferable Meta-Representations
• Kernel Conditional Density Operators
• Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data