Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

Abstract

The ambiguous annotation criteria lead to divergence of Chinese Word Segmentation (CWS) datasets in various granularities. Multi-criteria Chinese word segmentation aims to capture various annotation criteria among datasets and leverage their common underlying knowledge. In this paper, we propose a domain adaptive segmenter to exploit diverse criteria of various datasets. Our model is based on Bidirectional Encoder Representations from Transformers (BERT), which is responsible for introducing open-domain knowledge. Private and shared projection layers are proposed to capture domain-specific knowledge and common knowledge, respectively. We also optimize computational efficiency via distillation, quantization, and compiler optimization. Experiments show that our segmenter outperforms the previous state of the art (SOTA) models on 10 CWS datasets with superior efficiency.

arXiv commentAccepted at COLING 2020

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Do Deep Nets Really Need to be Deep?
• Compressing Deep Convolutional Networks using Vector Quantization
• Assessing BERT's Syntactic Abilities
• Subword Encoding in Lattice LSTM for Chinese Word Segmentation
• Deep Multi-Task Learning with Shared Memory
• Switch-LSTMs for Multi-Criteria Chinese Word Segmentation

Cited by in corpus (13)

• Glyce: Glyph-vectors for Chinese Character Representations
• Unified Multi-Criteria Chinese Word Segmentation with BERT
• Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities
• Attention Is All You Need for Chinese Word Segmentation
• N-LTP: An Open-source Neural Language Technology Platform for Chinese
• RethinkCWS: Is Chinese Word Segmentation a Solved Task?
• Dual-View Distilled BERT for Sentence Embedding
• fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP
• A New Clustering neural network for Chinese word segmentation
• Coupling Distant Annotation and Adversarial Training for Cross-Domain Chinese Word Segmentation
• End-to-End Chinese Parsing Exploiting Lexicons
• A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder
• Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application