25 citations · 120 across the 35 of their papers we have counts for
35 papers · 1 filter
PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
Xinbei Ma, Yeyun Gong, Pengcheng He +2
Based on the remarkable achievements of pre-trained language models in abstractive summarization, the copying mechanism has proved helpful by improving the factuality, stability, a…
Sentence Representation Learning with Generative Objective rather than Contrastive Objective
Bohong Wu, Hai Zhao
Though offering amazing contextualized token-level representations, current pre-trained language models take less attention on accurately acquiring sentence-level representation du…
Nested Named Entity Recognition as Holistic Structure Parsing
Yifei Yang, Zuchao Li, Hai Zhao
As a fundamental natural language processing task and one of core knowledge extraction techniques, named entity recognition (NER) is widely used to extract information from texts f…
Generative or Contrastive? Phrase Reconstruction for Better Sentence Representation Learning
Bohong Wu, Hai Zhao
Though offering amazing contextualized token-level representations, current pre-trained language models actually take less attention on acquiring sentence-level representation duri…
Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model
Hongjiang Jing, Zuchao Li, Hai Zhao +1
Aspect-based sentiment analysis (ABSA) task consists of three typical subtasks: aspect term extraction, opinion term extraction, and sentiment polarity classification. These three…
Unsupervised Full Constituency Parsing with Neighboring Distribution Divergence
Letian Peng, Zuchao Li, Hai Zhao
Unsupervised constituency parsing has been explored much but is still far from being solved. Conventional unsupervised constituency parser is only able to capture the unlabeled str…
Multi-tasking Dialogue Comprehension with Discourse Parsing
Yuchen He, Zhuosheng Zhang, Hai Zhao
Multi-party dialogue machine reading comprehension (MRC) raises an even more challenging understanding goal on dialogue with more than two involved speakers, compared with the trad…
Contextualized Semantic Distance between Highly Overlapped Texts
Letian Peng, Zuchao Li, Hai Zhao
Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. Better evaluation of the semantic distance…
Sparse Fuzzy Attention for Structured Sentiment Analysis
Letian Peng, Zuchao Li, Hai Zhao
Attention scorers have achieved success in parsing tasks like semantic and syntactic dependency parsing. However, in tasks modeled into parsing, like structured sentiment analysis,…
Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension
Yiyang Li, Hai Zhao
Multi-party dialogue machine reading comprehension (MRC) brings tremendous challenge since it involves multiple speakers at one dialogue, resulting in intricate speaker information…
Span Fine-tuning for Pre-trained Language Models
Rongzhou Bao, Zhuosheng Zhang, Hai Zhao
Pre-trained language models (PrLM) have to carefully manage input units when training on a very large text with a vocabulary consisting of millions of words. Previous works have sh…
Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension
Xinbei Ma, Zhuosheng Zhang, Hai Zhao
Multi-party multi-turn dialogue comprehension brings unprecedented challenges on handling the complicated scenarios from multiple speakers and criss-crossed discourse relationship…
Smoothing Dialogue States for Open Conversational Machine Reading
Zhuosheng Zhang, Siru Ouyang, Hai Zhao +2
Conversational machine reading (CMR) requires machines to communicate with humans through multi-turn interactions between two salient dialogue states of decision making and questio…
Unsupervised Open-Domain Question Answering
Pengfei Zhu, Xiaoguang Li, Jian Li +1
Open-domain Question Answering (ODQA) has achieved significant results in terms of supervised learning manner. However, data annotation cannot also be irresistible for its huge dem…
Dialogue-oriented Pre-training
Yi Xu, Hai Zhao
Pre-trained language models (PrLM) has been shown powerful in enhancing a broad range of downstream tasks including various dialogue related ones. However, PrLMs are usually traine…
Cross-lingual Transferring of Pre-trained Contextualized Language Models
Zuchao Li, Kevin Parnow, Hai Zhao +4
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons:…
Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
Bohong Wu, Zhuosheng Zhang, Hai Zhao
Multi-hop reading comprehension (MHRC) requires not only to predict the correct answer span in the given passage, but also to provide a chain of supporting evidences for reasoning…
Not All Attention Is All You Need
Hongqiu Wu, Hai Zhao, Min Zhang
Beyond the success story of pre-trained language models (PrLMs) in recent natural language processing, they are susceptible to over-fitting due to unusual large model size. To this…
Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrifice
Rongzhou Bao, Jiayi Wang, Hai Zhao
Pre-trained contextualized language models (PrLMs) have led to strong performance gains in downstream natural language understanding tasks. However, PrLMs can still be easily foole…
Pre-training Universal Language Representation
Yian Li, Hai Zhao
Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific levels of linguistic units. This work introd…
Grammatical Error Correction as GAN-like Sequence Labeling
Kevin Parnow, Zuchao Li, Hai Zhao
In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is…
Structural Pre-training for Dialogue Comprehension
Zhuosheng Zhang, Hai Zhao
Pre-trained language models (PrLMs) have demonstrated superior performance due to their strong ability to learn universal language representations from self-supervised pre-training…
Head-driven Phrase Structure Parsing in O() Time Complexity
Zuchao Li, Junru Zhou, Hai Zhao +1
Constituent and dependency parsing, the two classic forms of syntactic parsing, have been found to benefit from joint training and decoding under a uniform formalism, Head-driven P…
Neural Unsupervised Semantic Role Labeling
Kashif Munir, Hai Zhao, Zuchao Li
The task of semantic role labeling (SRL) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in…