# Boltzmann Machine

A Boltzmann machine is a stochastic recurrent neural network that learns probability distributions over binary states, using energy-based dynamics inspired by statistical mechanics. It is a Markov random field and an instance of the Sherrington-Kirkpatrick model with an external field.

A Boltzmann machine is a type of stochastic recurrent [neural network](https://www.wikiprompt.org/wiki/neural-network) that learns to represent and sample from probability distributions over binary output states. It is named after the Boltzmann distribution from statistical mechanics, which governs the probability of each network state based on its energy. The model is also classified as a Markov random field and is a specific instance of the Sherrington-Kirkpatrick model with an external field, also known as a stochastic Ising model. It was heavily promoted in cognitive science and [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) communities by Geoffrey Hinton, Terry Sejnowski, and Yann LeCun as part of the broader class of energy-based models, where the Hamiltonian of a spin glass serves as the energy function defining the learning task.

Boltzmann machines are theoretically notable for the locality and Hebbian nature of their training algorithm, which follows Hebb's rule, and for their parallelism and resemblance to simple physical processes. However, Boltzmann machines with unconstrained connectivity have not proven useful for practical problems in machine learning or inference. When connectivity is properly constrained, such as in restricted Boltzmann machines, learning becomes efficient enough for practical applications.

## Structure

A Boltzmann machine consists of a network of binary units, each producing a state of either 0 or 1. The overall network has a global energy function, identical in form to that of Hopfield networks and Ising models, defined as:

E = - (sum over i<j of w_ij * s_i * s_j + sum over i of theta_i * s_i)

Here, w_ij is the connection strength between unit j and unit i, s_i is the binary state of unit i, and theta_i is the bias of unit i, with -theta_i representing the activation threshold. The weights are often represented as a symmetric matrix W with zeros along the diagonal.

## Unit State Probability

The difference in global energy resulting from a single unit i being off versus on, denoted as Delta E_i, is given by the sum of weights from connected units plus the bias. This energy difference relates to the probabilities of the two states through the Boltzmann factor, where the probability of a state is proportional to exp(-E / (k_B T)), with k_B being the Boltzmann constant and T an artificial temperature parameter. This relationship allows the network to update units stochastically, favoring lower-energy configurations.

## Training Algorithm

The training of a Boltzmann machine aims to adjust the weights and biases so that the network's equilibrium distribution matches a target distribution over visible units. The learning rule is local and Hebbian: the change in a weight is proportional to the difference between the correlation of the two units when the network is clamped to data and when it is free-running. This contrastive learning procedure relies on sampling from the model's distribution, often using Markov chain Monte Carlo methods. The algorithm's locality makes it biologically plausible, but unconstrained networks suffer from slow convergence and poor scalability.

## Practical Variants

To address the inefficiency of fully connected Boltzmann machines, researchers introduced restricted Boltzmann machines (RBMs), which constrain connectivity to two layers - visible and hidden - with no intra-layer connections. This constraint enables more efficient training using contrastive divergence. Stacked RBMs form the basis of deep belief networks, which were influential in the early development of [deep-learning](https://www.wikiprompt.org/wiki/deep-learning). These variants have been applied to tasks such as dimensionality reduction, feature learning, and collaborative filtering, though they have largely been superseded by other architectures in many domains.

## Legacy and Influence

The Boltzmann machine contributed foundational concepts to [generative-ai](https://www.wikiprompt.org/wiki/generative-ai) and energy-based models. Its stochastic dynamics and probabilistic interpretation influenced later developments in neural network research, including the use of latent variables and sampling-based inference. While not widely used in contemporary large-scale systems like [large language models](https://www.wikiprompt.org/wiki/large-language-model), its theoretical insights remain relevant in understanding probabilistic graphical models and unsupervised learning. The model's name persists in the literature as a canonical example of a stochastic recurrent network and a bridge between statistical physics and [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence).

---
Source: https://www.wikiprompt.org/wiki/boltzmann-machine
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-07T02:36:04.687636+00:00