Alternating Projections for Learning with Expectation Constraints

Abstract

We present an objective function for learning with unlabeled data that utilizes auxiliary expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than generalized expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to generalized expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning.

Appears in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI2009)

References in corpus (1)

• Multi-View Learning over Structured and Non-Identical Outputs

Cited by in corpus (9)

• $\propto$SVM for learning with label proportions
• Feature-Rich Part-of-speech Tagging for Morphologically Complex Languages: Application to Bulgarian
• Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs
• Information-theoretic Semi-supervised Metric Learning via Entropy Regularization
• Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning
• Context-aware learning for generative models
• Conditional Random Field Autoencoders for Unsupervised Structured Prediction
• SERAPH: Semi-supervised Metric Learning Paradigm with Hyper Sparsity
• Point-to-set distance functions for weakly supervised segmentation