Featurized Density Ratio Estimation
arXiv:2107.02212
Abstract
Density ratio estimation serves as an important technique in the unsupervised machine learning toolbox. However, such ratios are difficult to estimate for complex, high-dimensional data, particularly when the densities of interest are sufficiently different. In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space. Empirically, we demonstrate the efficacy of our approach in a variety of downstream tasks that require access to accurate density ratios such as mutual information estimation, targeted sampling in deep generative models, and classification with data augmentation.
arXiv commentFirst two authors contributed equally
References in corpus (21)
- Learning Transferable Features with Deep Adaptation Networks
- Variational Inference with Normalizing Flows
- Matching Networks for One Shot Learning
- Data Augmentation Generative Adversarial Networks
- Density estimation using Real NVP
- f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
- MADE: Masked Autoencoder for Distribution Estimation
- Masked Autoregressive Flow for Density Estimation
- Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
- Improving Variational Inference with Inverse Autoregressive Flow
- On Variational Bounds of Mutual Information
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
- i-RevNet: Deep Invertible Networks
- Learning Likelihoods with Conditional Normalizing Flows
- Understanding the Limitations of Variational Mutual Information Estimators
- A RAD approach to deep mixture models
- Bayesian Semisupervised Learning with Deep Generative Models
- MintNet: Building Invertible Neural Networks with Masked Convolutions
- Constructive Setting of the Density Ratio Estimation Problem and its Rigorous Solution
- Telescoping Density-Ratio Estimation