A Stochastic Finite-State Word-Segmentation Algorithm for Chinese

Abstract

We present a stochastic finite-state model for segmenting Chinese text into dictionary entries and productively derived words, and providing pronunciations for these words; the method incorporates a class-based model in its treatment of personal names. We also evaluate the system's performance, taking into account the fact that people often do not agree on a single segmentation.

To appear in Proceedings of ACL-94

Cited by in corpus (12)

• Unsupervised Language Acquisition
• Automatic Discovery of Non-Compositional Compounds in Parallel Data
• Speech Recognition by Composition of Weighted Finite Automata
• Mostly-Unsupervised Statistical Segmentation of Japanese Kanji Sequences
• Use of Weighted Finite State Transducers in Part of Speech Tagging
• A realistic and robust model for Chinese word segmentation
• Towards Accurate Word Segmentation for Chinese Patents
• Annotation of Chinese Predicate Heads and Relevant Elements
• Algorithms for Speech Recognition and Language Processing
• Word Segmentation as Graph Partition
• An Iterative Algorithm to Build Chinese Language Models
• Building Chinese Lexicons from Scratch by Unsupervised Short Document Self-Segmentation