607 citations · 914 across the 15 of their papers we have counts for
10 papers · 1 filter
On the Transformer Growth for Progressive BERT Training
Xiaotao Gu, Liyuan Liu, Hongkun Yu +3
Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model a…
Very Deep Transformers for Neural Machine Translation
Xiaodong Liu, Kevin Duh, Liyuan Liu +1
We explore the application of very deep Transformer models for Neural Machine Translation (NMT). Using a simple yet effective initialization technique that stabilizes training, we…
Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling
Ouyu Lan, Xiao Huang, Bill Yuchen Lin +3
Sequence labeling is a fundamental framework for various natural language processing problems. Its performance is largely influenced by the annotation quality and quantity in super…
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Zihan Wang, Jingbo Shang, Liyuan Liu +3
Everyone makes mistakes. So do human annotators when curating labels for named entity recognition (NER). Such label mistakes might hurt model training and interfere model compariso…
Facet-Aware Evaluation for Extractive Summarization
Yuning Mao, Liyuan Liu, Qi Zhu +2
Commonly adopted metrics for extractive summarization focus on lexical overlap at the token level. In this paper, we present a facet-aware evaluation setup for better assessment of…
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Yujin Yuan, Liyuan Liu, Siliang Tang +5
Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations. However, the generated train…
Learning Named Entity Tagger using Domain-Specific Dictionary
Jingbo Shang, Liyuan Liu, Xiang Ren +3
Recent advances in deep neural models allow us to build reliable named entity recognition (NER) systems without handcrafting features. However, such methods require large amounts o…
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Liyuan Liu, Xiang Ren, Jingbo Shang +2
Many efforts have been made to facilitate natural language processing tasks with pre-trained language models (LMs), and brought significant improvements to various applications. To…
Empower Sequence Labeling with Task-Aware Neural Language Model
Liyuan Liu, Jingbo Shang, Frank F. Xu +4
Linguistic sequence labeling is a general modeling approach that encompasses a variety of problems, such as part-of-speech tagging and named entity recognition. Recent advances in…
Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach
Liyuan Liu, Xiang Ren, Qi Zhu +4
Relation extraction is a fundamental task in information extraction. Most existing methods have heavy reliance on annotations labeled by human experts, which are costly and time-co…