Autoassociative memory, also known as auto-association memory or an autoassociation network, is any type of memory that can retrieve a piece of data from only a tiny sample of itself. This capability makes it effective for de-noising inputs, removing interference, and determining whether a given input is "known" or "unknown" based on stored patterns. In artificial neural networks, examples include variational autoencoders, denoising autoencoders, and Hopfield networks. In computer memory systems, the concept is also referred to as content-addressable memory (CAM). The network is said to recognize a "known" vector if it produces an output activation pattern identical to one of the stored vectors.
Background
Standard data storage is indexed by positional memory addresses, which are used for retrieval. Autoassociative memories, in contrast, are organized in a graph-like structure where connection weights reflect the inherent associative connections between memories. This allows queries using a memory fragment as a key, retrieving that memory and closely connected ones simultaneously. Hopfield networks have been shown to act as autoassociative memory because they can remember data by observing a portion of it.
Iterative Autoassociative Nets
In some cases, an autoassociative net does not reproduce a stored pattern on the first attempt, but if the output is fed back as input, the stored pattern emerges. Three further types exist: the recurrent linear auto-associator, the Brain-State-in-a-Box net, and the discrete Hopfield net. The Hopfield network is the most well-known example of an autoassociative memory.
Hopfield Network
Hopfield networks serve as content-addressable (associative) memory systems with binary threshold nodes. They have been shown to act as autoassociative memories because they can recall complete patterns from partial input. The network's dynamics converge to stored attractors, enabling pattern completion and noise removal.
Heteroassociative Memory
Heteroassociative memories, by contrast, recall an associated piece of data from one category upon presentation of data from another category. For example, the pattern "banana" might trigger recall of the different pattern "monkey." Bidirectional associative memories (BAM) are artificial neural networks long used for heteroassociative recall.
Example
The sentence fragments below are sufficient for most English-speaking adults to recall the missing information:
- "To be or not to be, that is _____."
- "I came, I saw, _____."
Many readers will realize the missing information is:
- "To be or not to be, that is the question."
- "I came, I saw, I conquered."
This demonstrates the capability of autoassociative networks to recall the whole from some of its parts.
Applications
Autoassociative memories are widely used in Machine learning for tasks such as image denoising, data compression, and anomaly detection. Deep learning architectures like neural networks incorporate autoassociative principles in generative models and large language models for context completion. In hardware, content-addressable memory is implemented in AMD and Intel processors for high-speed lookup operations.
Related Concepts
Autoassociative memory is closely related to associative memory and content-addressable memory. In Artificial intelligence, it underpins techniques like denoising autoencoders and Hopfield networks. The concept also connects to heteroassociative memory and bidirectional associative memory.