Not All Neural Embeddings are Born Equal

Abstract

Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status.

4 pages plus 1 page of references

References in corpus (3)

• Sequence to Sequence Learning with Neural Networks
• Natural Language Processing (almost) from Scratch
• From Frequency to Meaning: Vector Space Models of Semantics

Cited by in corpus (1)

• Word and Document Embeddings based on Neural Network Approaches