# Neural Turing machines

A neural Turing machine (NTM) is a recurrent neural network with external memory, introduced by Alex Graves et al. in 2014, combining neural pattern matching with algorithmic programmability.

A neural Turing machine (NTM) is a recurrent neural network model that emulates a Turing machine. It was introduced by Alex Graves and colleagues in 2014. NTMs combine the fuzzy pattern matching capabilities of neural networks with the algorithmic power of programmable computers, enabling them to learn simple algorithms from examples.

The architecture consists of a neural network controller coupled to external memory resources. The controller interacts with the memory through attentional mechanisms, and the entire system is differentiable end-to-end, allowing optimization via gradient descent. With a long short-term memory (LSTM) controller, an NTM can infer algorithms such as copying, sorting, and associative recall from examples alone.

## Architecture and Operation

The NTM's controller is typically a recurrent neural network, such as an LSTM or a feedforward network. It receives input and produces output while also generating read and write operations on an external memory matrix. The memory is addressed using content-based and location-based attention, which allows the network to focus on specific memory locations. The differentiable nature of these operations enables training through backpropagation.

## Historical Context and Impact

The 2014 paper by Graves et al. was published at a time when deep learning was rapidly advancing. NTMs were among the first models to explicitly integrate external memory into neural networks, influencing later architectures like the [transformer](https://www.wikiprompt.org/wiki/transformer) and [large language models](https://www.wikiprompt.org/wiki/large-language-model). The concept drew inspiration from the Turing machine and aimed to bridge the gap between neural networks and symbolic computation.

## Implementations and Challenges

The original authors did not release their source code. The first stable open-source implementation appeared in 2018 at the 27th International Conference on Artificial Neural Networks, receiving a best-paper award. Other open-source implementations exist, but as of 2018 they were not sufficiently stable for production use. Developers reported issues such as gradients becoming NaN during training for unknown reasons, slow convergence, or lack of reported learning speed.

## Extensions and Legacy

Differentiable neural computers are an outgrowth of NTMs, incorporating more sophisticated attention mechanisms to control memory activity and improve performance. The NTM concept has also influenced research in [neural networks](https://www.wikiprompt.org/wiki/neural-network) and [machine learning](https://www.wikiprompt.org/wiki/machine-learning) more broadly, particularly in areas requiring memory-augmented models.

## See Also

- [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence)
- [deep-learning](https://www.wikiprompt.org/wiki/deep-learning)
- [recurrent-neural-network](https://www.wikiprompt.org/wiki/recurrent-neural-network)
- [attention-mechanism](https://www.wikiprompt.org/wiki/attention-mechanism)

---
Source: https://www.wikiprompt.org/wiki/neural-turing-machines
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-07T02:34:00.855731+00:00