1.5k citations · 5.2k across the 127 of their papers we have counts for
22 papers · 1 filter
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
Linyang Li, Yunfan Shao, Demin Song +2
Adversarial attacks in texts are mostly substitution-based methods that replace words or characters in the original texts to achieve success attacks. Recent methods use pre-trained…
Finding Sparse Structures for Domain Specific Neural Machine Translation
Jianze Liang, Chengqi Zhao, Mingxuan Wang +2
Neural machine translation often adopts the fine-tuning approach to adapt to specific domains. However, nonrestricted fine-tuning can easily degrade on the general domain and over-…
Text Information Aggregation with Centrality Attention
Jingjing Gong, Hang Yan, Yining Zheng +2
A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves aggregation process of combining the representations o…
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
Yiran Chen, Pengfei Liu, Ming Zhong +4
Neural network-based models augmented with unsupervised pre-trained knowledge have achieved impressive performance on text summarization. However, most existing evaluation methods…
Pre-training with Meta Learning for Chinese Word Segmentation
Zhen Ke, Liang Shi, Songtao Sun +3
Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS). However, PTMs used in previous works usually adopt language modeling as pre…
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
Zehui Lin, Xiao Pan, Mingxuan Wang +4
We investigate the following question for machine translation (MT): can we develop a single universal MT model to serve as the common seed and obtain derivative and improved models…
CoLAKE: Contextualized Language and Knowledge Embedding
Tianxiang Sun, Yunfan Shao, Xipeng Qiu +4
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained…
BERT for Monolingual and Cross-Lingual Reverse Dictionary
Hang Yan, Xiaonan Li, Xipeng Qiu
Reverse dictionary is the task to find the proper target word given the word description. In this paper, we tried to incorporate BERT into this task. However, since BERT is based o…
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Wei Zhu, Xipeng Qiu, Yuan Ni +1
Although BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models, it seems that no consensus can be reached…
fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP
Zhichao Geng, Hang Yan, Xipeng Qiu +1
We present fastHan, an open-source toolkit for four basic tasks in Chinese natural language processing: Chinese word segmentation (CWS), Part-of-Speech (POS) tagging, named entity…
AutoTrans: Automating Transformer Design via Reinforced Architecture Search
Wei Zhu, Xiaoling Wang, Xipeng Qiu +2
Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, espe…
Improving Image Captioning with Better Use of Captions
Zhan Shi, Xu Zhou, Xipeng Qiu +1
Image captioning is a multimodal problem that has drawn extensive attention in both the natural language processing and computer vision community. In this paper, we present a novel…
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Qipeng Guo, Zhijing Jin, Xipeng Qiu +3
Two important tasks at the intersection of knowledge graphs and natural language processing are graph-to-text (G2T) and text-to-graph (T2G) conversion. Due to the difficulty and hi…
Relation of the Relations: A New Paradigm of the Relation Extraction Problem
Zhijing Jin, Yongyi Yang, Xipeng Qiu +1
In natural language, often multiple entities appear in the same text. However, most previous works in Relation Extraction (RE) limit the scope to identifying the relation between t…
FLAT: Chinese NER Using Flat-Lattice Transformer
Xiaonan Li, Hang Yan, Xipeng Qiu +1
Recently, the character-word lattice structure has been proved to be effective for Chinese named entity recognition (NER) by incorporating the word information. However, since the…
Heterogeneous Graph Neural Networks for Extractive Document Summarization
Danqing Wang, Pengfei Liu, Yining Zheng +2
As a crucial step in extractive document summarization, learning cross-sentence relations has been explored by a plethora of approaches. An intuitive way is to put them in the grap…
Extractive Summarization as Text Matching
Ming Zhong, Pengfei Liu, Yiran Chen +3
This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. Instead of following the commonly used framework of extracting sentence…
Unified Multi-Criteria Chinese Word Segmentation with BERT
Zhen Ke, Liang Shi, Erli Meng +3
Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word boundaries in a Chinese sentence composed of continuous characters while multiple segmentation criteria exist.…
TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding
Linyang Li, Xipeng Qiu
Gradient-based adversarial training is widely used in improving the robustness of neural networks, while it cannot be easily adapted to natural language processing tasks since the…
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Linyang Li, Ruotian Ma, Qipeng Guo +2
Adversarial attacks for discrete data (such as texts) have been proved significantly more challenging than continuous data (such as images) since it is difficult to generate advers…
Pre-trained Models for Natural Language Processing: A Survey
Xipeng Qiu, Tianxiang Sun, Yige Xu +3
Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP.…
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
Yige Xu, Xipeng Qiu, Ligao Zhou +1
Fine-tuning pre-trained language models like BERT has become an effective way in NLP and yields state-of-the-art results on many downstream tasks. Recent studies on adapting BERT t…