One-shot and few-shot learning of word embeddings

Abstract

Standard deep learning systems require thousands or millions of examples to learn a concept, and cannot integrate new concepts easily. By contrast, humans have an incredible ability to do one-shot or few-shot learning. For instance, from just hearing a word used in a sentence, humans can infer a great deal about it, by leveraging what the syntax and semantics of the surrounding words tells us. Here, we draw inspiration from this to highlight a simple technique by which deep recurrent networks can similarly exploit their prior knowledge to learn a useful representation for a new word from little data. This could make natural language processing systems much more flexible, by allowing them to learn continually from the new words they encounter.

arXiv comment15 pages, 7 figures, under review as a conference paper at ICLR 2018

References in corpus (1)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Cited by in corpus (6)

• Transforming task representations to perform novel tasks
• Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks
• Learning from similarity and information extraction from structured documents
• Antonym-Synonym Classification Based on New Sub-space Embeddings
• Zero-shot task adaptation by homoiconic meta-mapping
• Diagnosis and Analysis of Celiac Disease and Environmental Enteropathy on Biopsy Images using Deep Learning Approaches