Nonlinear Models Using Dirichlet Process Mixtures

Abstract

We introduce a new nonlinear model for classification, in which we model the joint distribution of response variable, y, and covariates, x, non-parametrically using Dirichlet process mixtures. We keep the relationship between y and x linear within each component of the mixture. The overall relationship becomes nonlinear if the mixture contains more than one component. We use simulated data to compare the performance of this new approach to a simple multinomial logit (MNL) model, an MNL model with quadratic terms, and a decision tree model. We also evaluate our approach on a protein fold classification problem, and find that our model provides substantial improvement over previous methods, which were based on Neural Networks (NN) and Support Vector Machines (SVM). Folding classes of protein have a hierarchical structure. We extend our method to classification problems where a class hierarchy is available. We find that using the prior information regarding the hierarchical structure of protein folds can result in higher predictive accuracy.

References in corpus (1)

• The Short-Cut Metropolis Method

Cited by in corpus (26)

• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• Breaking Sticks and Ambiguities with Adaptive Skip-gram
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
• A Practical Introduction to Bayesian Estimation of Causal Effects: Parametric and Nonparametric Approaches
• The supervised hierarchical Dirichlet process
• Attention over Parameters for Dialogue Systems
• Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
• Multiple Imputation of Missing Categorical and Continuous Values via Bayesian Mixture Models with Local Dependence
• Customized training with an application to mass spectrometric imaging of cancer tissue
• Bayesian learning of joint distributions of objects
• MoEL: Mixture of Empathetic Listeners
• Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models
• Bayesian nonparametric generative models for causal inference with missing at random covariates
• Density Estimation and Classification via Bayesian Nonparametric Learning of Affine Subspaces
• Non-Gaussian Discriminative Factor Models via the Max-Margin Rank-Likelihood
• Selecting Data Adaptive Learner from Multiple Deep Learners using Bayesian Networks
• A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms
• Outcome identification in electronic health records using predictions from an enriched Dirichlet process mixture
• Bayesian Nonparametric Density Autoregression with Lag Selection
• Bayesian density regression for discrete outcomes
• Fast Sampling for Bayesian Max-Margin Models
• Classification Algorithm of Speech Data of Parkinsons Disease Based on Convolution Sparse Kernel Transfer Learning with Optimal Kernel and Parallel Sample Feature Selection
• Stochastic Search with an Observable State Variable
• Towards Bursting Filter Bubble via Contextual Risks and Uncertainties
• CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction
• Learning Densities Conditional on Many Interacting Features