Diversified Hidden Markov Models for Sequential Labeling
arXiv:1904.03170 · doi:10.1109/TKDE.2015.2433262
Abstract
Labeling of sequential data is a prevalent meta-problem for a wide range of real world applications. While the first-order Hidden Markov Models (HMM) provides a fundamental approach for unsupervised sequential labeling, the basic model does not show satisfying performance when it is directly applied to real world problems, such as part-of-speech tagging (PoS tagging) and optical character recognition (OCR). Aiming at improving performance, important extensions of HMM have been proposed in the literatures. One of the common key features in these extensions is the incorporation of proper prior information. In this paper, we propose a new extension of HMM, termed diversified Hidden Markov Models (dHMM), which utilizes a diversity-encouraging prior over the state-transition probabilities and thus facilitates more dynamic sequential labellings. Specifically, the diversity is modeled by a continuous determinantal point process prior, which we apply to both unsupervised and supervised scenarios. Learning and inference algorithms for dHMM are derived. Empirical evaluations on benchmark datasets for unsupervised PoS tagging and supervised OCR confirmed the effectiveness of dHMM, with competitive performance to the state-of-the-art.
14 pages, 12 figures
References in corpus (10)
- Determinantal point processes for machine learning
- Determinantal Processes and Independence
- Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application
- Quantifying repulsiveness of determinantal point processes
- Learning the Parameters of Determinantal Point Process Kernels
- Expectation-Maximization for Learning Determinantal Point Processes
- Markov Determinantal Point Processes
- Determinantal Point Process Priors for Bayesian Variable Selection in Linear Regression
- Approximate Inference in Continuous Determinantal Point Processes
- Determinantal Clustering Processes - A Nonparametric Bayesian Approach to Kernel Based Semi-Supervised Clustering