Character-based Neural Machine Translation

Abstract

We introduce a neural machine translation model that views the input and output sentences as sequences of characters rather than words. Since word-level information provides a crucial source of bias, our input model composes representations of character sequences into representations of words (as determined by whitespace boundaries), and then these are translated using a joint attention/translation model. In the target language, the translation is modeled as a sequence of word vectors, but each word is generated one character at a time, conditional on the previous character generations in each word. As the representation and generation of words is performed at the character level, our model is capable of interpreting and generating unseen word forms. A secondary benefit of this approach is that it alleviates much of the challenges associated with preprocessing/tokenization of the source and target languages. We show that our model can achieve translation results that are on par with conventional word-based models.

References in corpus (1)

• Sequence to Sequence Learning with Neural Networks

Cited by in corpus (72)

• DyNet: The Dynamic Neural Network Toolkit
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Hierarchical Multiscale Recurrent Neural Networks
• Neural Language Correction with Character-Based Attention
• Sequence-Level Knowledge Distillation
• A Survey of Deep Learning Techniques for Neural Machine Translation
• Multiplicative LSTM for sequence modelling
• Attending to Characters in Neural Sequence Labeling Models
• Natural Language Understanding with Distributed Representation
• Bridging Neural Machine Translation and Bilingual Dictionaries
• Charagram: Embedding Words and Sentences via Character n-grams
• Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests
• SyntaxNet Models for the CoNLL 2017 Shared Task
• An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization
• Combining Discrete and Neural Features for Sequence Labeling
• Incorporating Discrete Translation Lexicons into Neural Machine Translation
• Zero-Resource Translation with Multi-Lingual Neural Machine Translation
• Character-Based Handwritten Text Transcription with Attention Networks
• Generative Concatenative Nets Jointly Learn to Write and Classify Reviews
• Who Needs Words? Lexicon-Free Speech Recognition
• Recurrent Neural Network Language Models for Open Vocabulary Event-Level Cyber Anomaly Detection
• Learning to Segment Inputs for NMT Favors Character-Level Processing
• Design Challenges in Named Entity Transliteration
• Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German
• Character-Word LSTM Language Models
• Exploring Benefits of Transfer Learning in Neural Machine Translation
• A Universal Parent Model for Low-Resource Neural Machine Translation Transfer
• Trainable Greedy Decoding for Neural Machine Translation
• Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network
• Lattice-Based Transformer Encoder for Neural Machine Translation
• Chunk-Based Bi-Scale Decoder for Neural Machine Translation
• A Latent Morphology Model for Open-Vocabulary Neural Machine Translation
• Character-based NMT with Transformer
• How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?
• Neural Machine Transliteration: Preliminary Results
• Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning
• Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
• BPE and CharCNNs for Translation of Morphology: A Cross-Lingual Comparison and Analysis
• Investigating an Effective Character-level Embedding in Korean Sentence Classification
• Low-rank passthrough neural networks
• Neural Name Translation Improves Neural Machine Translation
• Shared-Private Bilingual Word Embeddings for Neural Machine Translation
• PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents
• Generating new concepts with hybrid neuro-symbolic models
• An Efficient Character-Level Neural Machine Translation
• A Convolutional Neural Network for Language-Agnostic Source Code Summarization
• Improving Pre-Trained Multilingual Models with Vocabulary Expansion
• SubCharacter Chinese-English Neural Machine Translation with Wubi encoding
• The boundaries of meaning: a case study in neural machine translation
• Piecewise-constant Neural ODEs
• Neural Machine Translation via Binary Code Prediction
• Neural Machine Translation: A Review and Survey
• The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation
• Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation
• Finding Better Subword Segmentation for Neural Machine Translation
• Paraphrases as Foreign Languages in Multilingual Neural Machine Translation
• Using Interlinear Glosses as Pivot in Low-Resource Multilingual Machine Translation
• Neural Machine Translation with Characters and Hierarchical Encoding
• Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification
• Nonsymbolic Text Representation
• Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation
• Recurrent Neural Networks with Mixed Hierarchical Structures for Natural Language Processing
• Knowledge Efficient Deep Learning for Natural Language Processing
• What's Going On in Neural Constituency Parsers? An Analysis
• Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation
• Diversity by Phonetics and its Application in Neural Machine Translation
• Translating Questions into Answers using DBPedia n-triples
• Effective Character-augmented Word Embedding for Machine Reading Comprehension
• Stem-driven Language Models for Morphologically Rich Languages
• Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
• Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses
• JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to English