Achieving Human Parity on Automatic Chinese to English News Translation
arXiv:1803.05567
Abstract
Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether such systems can approach or achieve parity with human translations. In this paper, we first address the problem of how to define and accurately measure human parity in translation. We then describe Microsoft's machine translation system and measure the quality of its translations on the widely used WMT 2017 news translation task from Chinese to English. We find that our latest neural machine translation system has reached a new state-of-the-art, and that the translation quality is at human parity when compared to professional human translations. We also find that it significantly exceeds the quality of crowd-sourced non-professional translations.
References in corpus (4)
Cited by in corpus (136)
- A Simple Method for Commonsense Reasoning
- Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
- Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
- Phrase-Based & Neural Unsupervised Machine Translation
- Learning and Evaluating General Linguistic Intelligence
- Survey on reinforcement learning for language processing
- An Effective Approach to Unsupervised Machine Translation
- Understanding Back-Translation at Scale
- Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
- A Surprisingly Robust Trick for Winograd Schema Challenge
- Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize
- Universal Neural Machine Translation for Extremely Low Resource Languages
- Massively Multilingual Neural Machine Translation
- Sequence Length is a Domain: Length-based Overfitting in Transformer Models
- Understanding and Improving Lexical Choice in Non-Autoregressive Translation
- Impact of Artificial Intelligence on Businesses: from Research, Innovation, Market Deployment to Future Shifts in Business Models
- Self-Paced Learning for Neural Machine Translation
- Post-editese: an Exacerbated Translationese
- Controlling the Output Length of Neural Machine Translation
- Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation
- Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications
- Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval
- Context-Aware Self-Attention Networks
- scb-mt-en-th-2020: A Large English-Thai Parallel Corpus
- Exploiting Deep Representations for Neural Machine Translation
- Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019
- Multi-Head Attention with Disagreement Regularization
- Coreference Resolution as Query-based Span Prediction
- Non-Autoregressive Machine Translation with Auxiliary Regularization
- Unsupervised Neural Machine Translation with SMT as Posterior Regularization
- Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings
- A Study of Reinforcement Learning for Neural Machine Translation
- Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain
- Context-Aware Learning for Neural Machine Translation
- Explaining and Generalizing Back-Translation through Wake-Sleep
- Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
- Revisiting Round-Trip Translation for Quality Estimation
- A Variational Dirichlet Framework for Out-of-Distribution Detection
- Information Aggregation for Multi-Head Attention with Routing-by-Agreement
- Sequence to Sequence Mixture Model for Diverse Machine Translation
- Simple, Scalable Adaptation for Neural Machine Translation
- Feature reinforcement with word embedding and parsing information in neural TTS
- Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
- Hybrid Self-Attention Network for Machine Translation
- Multilingual Neural Machine Translation with Language Clustering
- Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
- Towards Understanding Neural Machine Translation with Word Importance
- Neural Machine Translation: Challenges, Progress and Future
- Efficient Deep Neural Networks
- Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers
- Don't Classify, Translate: Multi-Level E-Commerce Product Categorization Via Machine Translation
- Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
- TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
- Machine Translation: A Literature Review
- Improving Back-Translation with Uncertainty-based Confidence Estimation
- A Study of Multilingual Neural Machine Translation
- Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
- Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs
- Neural Machine Translation: A Review of Methods, Resources, and Tools
- Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation
- Language Graph Distillation for Low-Resource Machine Translation
- Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation
- Lattice Transformer for Speech Translation
- Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation
- Towards Making the Most of Context in Neural Machine Translation
- Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
- Accelerating Deep Learning Inference via Learned Caches
- Word Shape Matters: Robust Machine Translation with Visual Embedding
- The Effect of Translationese in Machine Translation Test Sets
- PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents
- Shared-Private Bilingual Word Embeddings for Neural Machine Translation
- Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
- Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation
- Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation
- Modeling Fluency and Faithfulness for Diverse Neural Machine Translation
- Optimal Completion Distillation for Sequence Learning
- Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
- Adaptive Nearest Neighbor Machine Translation
- Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation
- Learning Bilingual Sentence Embeddings via Autoencoding and Computing Similarities with a Multilayer Perceptron
- Automatic Testing and Improvement of Machine Translation
- Towards Computational Linguistics in Minangkabau Language: Studies on Sentiment Analysis and Machine Translation
- Capsule-Transformer for Neural Machine Translation
- Learning Language Specific Sub-network for Multilingual Machine Translation
- Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation
- The 2020s Political Economy of Machine Translation
- A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation
- Extractive Summary as Discrete Latent Variables
- Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
- Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation
- SubCharacter Chinese-English Neural Machine Translation with Wubi encoding
- IOT: Instance-wise Layer Reordering for Transformer Structures
- Efficient Bidirectional Neural Machine Translation
- Improving Bidirectional Decoding with Dynamic Target Semantics in Neural Machine Translation
- Facebook AI's WAT19 Myanmar-English Translation Task Submission
- Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer
- "Bilingual Expert" Can Find Translation Errors
- An Approach to Improve Robustness of NLP Systems against ASR Errors
- Fine-grained Human Evaluation of Transformer and Recurrent Approaches to Neural Machine Translation for English-to-Chinese
- Improving Neural Machine Translation by Bidirectional Training
- Faithful Target Attribute Prediction in Neural Machine Translation
- Iterative Domain-Repaired Back-Translation
- Neural Machine Translation: A Review and Survey
- Isomorphic Cross-lingual Embeddings for Low-Resource Languages
- Forward-Backward Decoding for Regularizing End-to-End TTS
- Towards a resource for multilingual lexicons: an MT assisted and human-in-the-loop multilingual parallel corpus with multi-word expression annotation
- Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation
- Structure-Invariant Testing for Machine Translation
- Hierarchical Document Encoder for Parallel Corpus Mining
- Context-Aware Monolingual Repair for Neural Machine Translation
- Automatic Classification of Human Translation and Machine Translation: A Study from the Perspective of Lexical Diversity
- Deliberation Model Based Two-Pass End-to-End Speech Recognition
- A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation
- Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation
- Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
- Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
- NEJM-enzh: A Parallel Corpus for English-Chinese Translation in the Biomedical Domain
- Depth Growing for Neural Machine Translation
- Language-Independent Representor for Neural Machine Translation
- Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit
- Distilling Translations with Visual Awareness
- Self-Guided Curriculum Learning for Neural Machine Translation
- Reciprocal Supervised Learning Improves Neural Machine Translation
- Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model
- Towards Making the Most of Dialogue Characteristics for Neural Chat Translation
- Big Bidirectional Insertion Representations for Documents
- Modeling Bilingual Conversational Characteristics for Neural Chat Translation
- Microsoft Research Asia's Systems for WMT19
- Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite
- Semantic Role Labeling with Iterative Structure Refinement
- GRET: Global Representation Enhanced Transformer
- Testing Machine Translation via Referential Transparency
- Learning to Detect Unacceptable Machine Translations for Downstream Tasks
- Neural Machine Translation with Error Correction
- Digital Normativity: A challenge for human subjectivization and free will
- Lexically Cohesive Neural Machine Translation with Copy Mechanism