Neural Turing Machines

Abstract

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

References in corpus (2)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks

Cited by in corpus (17)

• Attention-Based Models for Speech Recognition
• End-To-End Memory Networks
• Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
• Online and Linear-Time Attention by Enforcing Monotonic Alignments
• Control of Memory, Active Perception, and Action in Minecraft
• Learning to Transduce with Unbounded Memory
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• RobustFill: Neural Program Learning under Noisy I/O
• Associative Long Short-Term Memory
• Programming with a Differentiable Forth Interpreter
• Learning Program Embeddings to Propagate Feedback on Student Code
• How Important is Weight Symmetry in Backpropagation?
• Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
• Modeling citation worthiness by using attention-based bidirectional long short-term memory networks and interpretable models
• Genetic algorithms with DNN-based trainable crossover as an example of partial specialization of general search
• Contextualized Rewriting for Text Summarization
• Learning Permutation Invariant Representations using Memory Networks