206 citations · 624 across the 70 of their papers we have counts for
8 papers · 1 filter
Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Xiaoqi Jiao, Huating Chang, Yichun Yin +6
Knowledge distillation (KD) which transfers the knowledge from a large teacher model to a small student model, has been widely used to compress the BERT model recently. Besides the…
PPKE: Knowledge Representation Learning by Path-based Pre-training
Bin He, Di Zhou, Jing Xie +3
Entities may have complex interactions in a knowledge graph (KG), such as multi-step relationships, which can be viewed as graph contextual information of the entities. Traditional…
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning
Bin He, Xin Jiang, Jinghui Xiao +1
Recent studies on pre-trained language models have demonstrated their ability to capture factual knowledge and applications in knowledge-aware downstream tasks. In this work, we pr…
HyperText: Endowing FastText with Hyperbolic Geometry
Yudong Zhu, Di Zhou, Jinghui Xiao +3
Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text classifier based on shallo…
Learning to Detect Unacceptable Machine Translations for Downstream Tasks
Meng Zhang, Xin Jiang, Yang Liu +1
The field of machine translation has progressed tremendously in recent years. Even though the translation quality has improved significantly, current systems are still unable to pr…
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
Xin Liu, Jiefu Ou, Yangqiu Song +1
Implicit discourse relation classification is one of the most difficult parts in shallow discourse parsing as the relation prediction without explicit connectives requires the lang…
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi Liao, Xin Jiang, Qun Liu
Masked language model and autoregressive language model are two types of language models. While pretrained masked language models such as BERT overwhelm the line of natural languag…
Accurate Word Alignment Induction from Neural Machine Translation
Yun Chen, Yang Liu, Guanhua Chen +2
Despite its original goal to jointly learn to align and translate, prior researches suggest that Transformer captures poor word alignments through its attention mechanism. In this…