Towards Language Agnostic Universal Representations
arXiv:1809.08510
Abstract
When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in,even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we present a method to decouple the language from the problem by learning language agnostic representations and therefore allowing training a model in one language and applying to a different one in a zero shot fashion. We learn these representations by taking inspiration from linguistics and formalizing Universal Grammar as an optimization process (Chomsky, 2014; Montague, 1970). We demonstrate the capabilities of these representations by showing that the models trained on a single language using language agnostic representations achieve very similar accuracies in other languages.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Improved Training of Wasserstein GANs
- A large annotated corpus for learning natural language inference
- Learning to Generate Reviews and Discovering Sentiment
- Massively Multilingual Word Embeddings
- Learned in Translation: Contextualized Word Vectors
- Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews
- Unsupervised Neural Machine Translation
- Reading Wikipedia to Answer Open-Domain Questions
- Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
- Unsupervised Multilingual Word Embeddings