183 citations · 473 across the 6 of their papers we have counts for
9 papers
Question-Context Alignment and Answer-Context Dependencies for Effective Answer Sentence Selection
Minh Van Nguyen, Kishan KC, Toan Nguyen +3
Answer sentence selection (AS2) in open-domain question answering finds answer for a question by ranking candidate sentences extracted from web documents. Recent work exploits answ…
Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution
Toan Q. Nguyen, Kenton Murray, David Chiang
In this paper, we investigate the driving factors behind concatenation, a simple but effective data augmentation method for low-resource neural machine translation. Our experiments…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
Masked Language Model Scoring
Julian Salazar, Davis Liang, Toan Q. Nguyen +1
Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are co…
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation
Kenton Murray, Jeffery Kinnison, Toan Q. Nguyen +2
Neural sequence-to-sequence models, particularly the Transformer, are the state of the art in machine translation. Yet these neural networks are very sensitive to architecture and…
Transformers without Tears: Improving the Normalization of Self-Attention
Toan Q. Nguyen, Julian Salazar
We evaluate three simple, normalization-centric changes to improve Transformer training. First, we show that pre-norm residual connections (PreNorm) and smaller initializations ena…
Neural Machine Translation of Text from Non-Native Speakers
Antonios Anastasopoulos, Alison Lui, Toan Nguyen +1
Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data, especially when the system is trained only on clean data. In this paper, we show that…
Improving Lexical Choice in Neural Machine Translation
Toan Q. Nguyen, David Chiang
We explore two solutions to the problem of mistranslating rare words in neural machine translation. First, we argue that the standard output layer, which computes the inner product…
Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
Toan Q. Nguyen, David Chiang
We present a simple method to improve neural translation of a low-resource language pair using parallel data from a related, also low-resource, language pair. The method is based o…