1.2k citations · 7.6k across the 154 of their papers we have counts for
56 papers · 1 filter
Extract and Attend: Improving Entity Translation in Neural Machine Translation
Zixin Zeng, Rui Wang, Yichong Leng +4
While Neural Machine Translation(NMT) has achieved great progress in recent years, it still suffers from inaccurate translation of entities (e.g., person/organization name, locatio…
MolXPT: Wrapping Molecules with Text for Generative Pre-training
Zequn Liu, Wei Zhang, Yingce Xia +5
Generative pre-trained Transformer (GPT) has demonstrates its great success in natural language processing and related techniques have been adapted into molecular modeling. Conside…
ResiDual: Transformer with Dual Residual Connections
Shufang Xie, Huishuai Zhang, Junliang Guo +6
Transformer networks have become the preferred architecture for many tasks due to their state-of-the-art performance. However, the optimal way to implement residual connections in…
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Yisheng Xiao, Ruiyang Xu, Lijun Wu +4
Transformer-based autoregressive (AR) methods have achieved appealing performance for varied sequence-to-sequence generation tasks, e.g., neural machine translation, summarization,…
N-Gram Nearest Neighbor Machine Translation
Rui Lv, Junliang Guo, Rui Wang +3
Nearest neighbor machine translation augments the Autoregressive Translation~(AT) with -nearest-neighbor retrieval, by comparing the similarity between the token-level context r…
SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition
Yichong Leng, Xu Tan, Wenjie Liu +6
Error correction in automatic speech recognition (ASR) aims to correct those incorrect words in sentences generated by ASR models. Since recent ASR models usually have low word err…
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Renqian Luo, Liai Sun, Yingce Xia +4
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main…
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation
Kexun Zhang, Rui Wang, Xu Tan +4
It is difficult for non-autoregressive translation (NAT) models to capture the multi-modal distribution of target translations due to their conditional independence assumption, whi…
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi, Shufang Xie, Tao Qin +1
Multilingual Neural Machine Translation (MNMT) enables one system to translate sentences from multiple source languages to multiple target languages, greatly reducing deployment co…
Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling
Kaitao Song, Yichong Leng, Xu Tan +3
Sentence scoring aims at measuring the likelihood score of a sentence and is widely used in many natural language processing scenarios, like reranking, which is to select the best…
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
Yisheng Xiao, Lijun Wu, Junliang Guo +4
Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and na…
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
Liang He, Shizhuo Zhang, Lijun Wu +10
Understanding protein sequences is vital and urgent for biology, healthcare, and medicine. Labeling approaches are expensive yet time-consuming, while the amount of unlabeled data…
Discovering Drug-Target Interaction Knowledge from Biomedical Literature
Yutai Hou, Yingce Xia, Lijun Wu +6
The Interaction between Drugs and Targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedic…
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition
Yichong Leng, Xu Tan, Rui Wang +8
Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence, and can further reduce the word error rate (WER). Although multiple ca…
Analyzing and Mitigating Interference in Neural Architecture Search
Jin Xu, Xu Tan, Kaitao Song +5
Weight sharing is a popular approach to reduce the cost of neural architecture search (NAS) by reusing the weights of shared operators from previously trained child models. However…
A Survey on Low-Resource Neural Machine Translation
Rui Wang, Xu Tan, Renqian Luo +2
Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has…
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
Jin Xu, Xu Tan, Renqian Luo +4
While pre-trained language models (e.g., BERT) have achieved impressive results on different natural language processing tasks, they have large numbers of parameters and suffer fro…
Exploiting Adapters for Cross-lingual Low-resource Speech Recognition
Wenxin Hou, Han Zhu, Yidong Wang +4
Cross-lingual speech adaptation aims to solve the problem of leveraging multiple rich-resource languages to build models for a low-resource target language. Since the low-resource…
FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition
Yichong Leng, Xu Tan, Linchen Zhu +7
Error correction techniques have been used to refine the output sentences from automatic speech recognition (ASR) models and achieve a lower word error rate (WER) than original ASR…
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Zhen Wu, Lijun Wu, Qi Meng +5
Transformer architecture achieves great success in abundant natural language processing tasks. The over-parameterization of the Transformer model has motivated plenty of works to a…
IOT: Instance-wise Layer Reordering for Transformer Structures
Jinhua Zhu, Lijun Wu, Yingce Xia +5
With sequentially stacked self-attention, (optional) encoder-decoder attention, and feed-forward layers, Transformer achieves big success in natural language processing (NLP), and…
MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
Linghui Meng, Jin Xu, Xu Tan +3
In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR). MixSpeech trains an ASR model by taking…
Towards Interpretable Reasoning over Paragraph Effects in Situation
Mucheng Ren, Xiubo Geng, Tao Qin +2
We focus on the task of reasoning over paragraph effects in situation, which requires a model to understand the cause and effect described in a background paragraph, and apply the…
Knowledge-Aware Procedural Text Understanding with Multi-Stage Training
Zhihan Zhang, Xiubo Geng, Tao Qin +2
Procedural text describes dynamic state changes during a step-by-step natural process (e.g., photosynthesis). In this work, we focus on the task of procedural text understanding, w…