Online Bayesian Passive-Aggressive Learning

Abstract

Online Passive-Aggressive (PA) learning is an effective framework for performing max-margin online learning. But the deterministic formulation and estimated single large-margin model could limit its capability in discovering descriptive structures underlying complex data. This pa- per presents online Bayesian Passive-Aggressive (BayesPA) learning, which subsumes the online PA and extends naturally to incorporate latent variables and perform nonparametric Bayesian inference, thus providing great flexibility for explorative analysis. We apply BayesPA to topic modeling and derive efficient online learning algorithms for max-margin topic models. We further develop nonparametric methods to resolve the number of topics. Experimental results on real datasets show that our approaches significantly improve time efficiency while maintaining comparable results with the batch counterparts.

10 Pages. ICML 2014, Beijing, China

References in corpus (7)

• Expectation Propagation for approximate Bayesian inference
• Supervised Topic Models
• Bayesian Online Changepoint Detection
• Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring
• Sparse Stochastic Inference for Latent Dirichlet allocation
• Max-Margin Nonparametric Latent Feature Models for Link Prediction
• Hybrid Variational/Gibbs Collapsed Inference in Topic Models

Cited by in corpus (7)

• Big Learning with Bayesian Methods
• Online Deep Learning based on Auto-Encoder
• Fully Implicit Online Learning
• Online learning using multiple times weight updating
• p-Markov Gaussian Processes for Scalable and Expressive Online Bayesian Nonparametric Time Series Forecasting
• Fast Sampling for Bayesian Max-Margin Models
• Online Bayesian Collaborative Topic Regression