Efficiently Inducing Features of Conditional Random Fields

Abstract

Conditional Random Fields (CRFs) are undirected graphical models, a special case of which correspond to conditionally-trained finite state machines. A key advantage of these models is their great flexibility to include a wide array of overlapping, multi-granularity, non-independent features of the input. In face of this freedom, an important question that remains is, what features should be used? This paper presents a feature induction method for CRFs. Founded on the principle of constructing only those feature conjunctions that significantly increase log-likelihood, the approach is based on that of Della Pietra et al [1997], but altered to work with conditional rather than joint probabilities, and with additional modifications for providing tractability specifically for a sequence model. In comparison with traditional approaches, automated feature induction offers both improved accuracy and more than an order of magnitude reduction in feature count; it enables the use of richer, higher-order Markov models, and offers more freedom to liberally guess about which atomic features may be relevant to a task. The induction method applies to linear-chain CRFs, as well as to more arbitrary CRF structures, also known as Relational Markov Networks [Taskar & Koller, 2002]. We present experimental results on a named entity extraction task.

Appears in Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence (UAI2003)

References in corpus (1)

• The Use of Classifiers in Sequential Inference

Cited by in corpus (15)

• Efficient Markov Network Structure Discovery Using Independence Tests
• Feature-Rich Named Entity Recognition for Bulgarian Using Conditional Random Fields
• Efficient Learning of Sparse Conditional Random Fields for Supervised Sequence Labelling
• Learning Factor Graphs in Polynomial Time & Sample Complexity
• Texture Modelling with Nested High-order Markov-Gibbs Random Fields
• The IBMAP approach for Markov networks structure learning
• An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition
• Improving Named Entity Recognition with Attentive Ensemble of Syntactic Information
• Blankets Joint Posterior score for learning Markov network structures
• Markov random fields factorization with context-specific independences
• Efficient Independence-Based MAP Approach for Robust Markov Networks Structure Discovery
• Unravelling the Architecture of Membrane Proteins with Conditional Random Fields
• De-identification In practice
• Character Feature Engineering for Japanese Word Segmentation
• Contrastive Feature Induction for Efficient Structure Learning of Conditional Random Fields