Piecewise Training for Undirected Models

Abstract

For many large undirected models that arise in real-world applications, exact maximumlikelihood training is intractable, because it requires computing marginal distributions of the model. Conditional training is even more difficult, because the partition function depends not only on the parameters, but also on the observed input, requiring repeated inference over each training example. An appealing idea for such models is to independently train a local undirected classifier over each clique, afterwards combining the learned weights into a single global model. In this paper, we show that this piecewise method can be justified as minimizing a new family of upper bounds on the log partition function. On three natural-language data sets, piecewise training is more accurate than pseudolikelihood, and often performs comparably to global training using belief propagation.

Appears in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI2005)

Cited by in corpus (15)

• Learning Graphical Model Parameters with Approximate Marginal Inference
• Efficient piecewise training of deep structured models for semantic segmentation
• Deeply Learning the Messages in Message Passing Inference
• Exploring Context with Deep Structured models for Semantic Segmentation
• Products of Weighted Logic Programs
• Large-Scale Classification of Structured Objects using a CRF with Deep Class Embedding
• Mixture-of-Parents Maximum Entropy Markov Models
• What Cannot be Learned with Bethe Approximations
• Loss-sensitive Training of Probabilistic Conditional Random Fields
• A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images
• Conditional Random Field and Deep Feature Learning for Hyperspectral Image Segmentation
• Learning undirected models via query training
• A Parametric Top-View Representation of Complex Road Scenes
• Perturb-and-max-product: Sampling and learning in discrete energy-based models
• Learning to segment with image-level supervision