# Machine learning

Machine learning is a subfield of artificial intelligence focused on developing statistical algorithms that learn from data to make predictions or decisions without being explicitly programmed, encompassing supervised, unsupervised, and reinforcement learning.

Machine learning is a subfield of artificial intelligence in which computer systems improve their performance on a task by learning statistical patterns from data, rather than following rules that are explicitly programmed by hand. It underlies most contemporary [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) applications, from spam filters to [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s.

The term is generally attributed to [Arthur Samuel](https://www.wikiprompt.org/wiki/arthur-samuel), who used it in 1959 to describe a checkers-playing program that improved through self-play.

## Paradigms

Machine learning is commonly divided into three paradigms. In [supervised-learning](https://www.wikiprompt.org/wiki/supervised-learning), a model learns from examples paired with correct labels, such as images tagged with object categories. In [unsupervised-learning](https://www.wikiprompt.org/wiki/unsupervised-learning), a model finds structure in unlabeled data, for example by clustering similar items together. In [reinforcement-learning](https://www.wikiprompt.org/wiki/reinforcement-learning), an agent learns by taking actions in an environment and receiving rewards or penalties, a paradigm that produced landmark results such as AlphaGo. A fourth approach, [self-supervised-learning](https://www.wikiprompt.org/wiki/self-supervised-learning), generates its own training signal from unlabeled data, for instance by predicting a hidden word from its context, and is the method behind the pretraining of modern language models.

## Core methods

Most machine learning models are trained by defining a [loss-function](https://www.wikiprompt.org/wiki/loss-function) that measures how wrong the model's predictions are, then adjusting the model's parameters to reduce that loss through [gradient-descent](https://www.wikiprompt.org/wiki/gradient-descent) or a related optimization procedure. Statistical approaches that predate the deep learning era, such as support vector machines developed by [vladimir-vapnik](https://www.wikiprompt.org/wiki/vladimir-vapnik) and colleagues, remain in use for smaller or more interpretable problems. Since the 2010s, [neural-network](https://www.wikiprompt.org/wiki/neural-network) models, particularly [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) architectures, have become the dominant method for tasks involving images, audio, and text, given sufficient training data and compute.

## Practical concerns

A central challenge in machine learning is [overfitting](https://www.wikiprompt.org/wiki/overfitting), where a model memorizes patterns specific to its training data rather than learning patterns that generalize to new examples. Practitioners address this through techniques such as regularization, cross-validation, and holding out test data. The quality and scale of training data, along with the choice of model architecture, are typically larger determinants of real-world performance than any single algorithmic trick.

## Adoption

Machine learning moved from an academic discipline to a mainstream engineering practice over the 2010s, driven partly by publicly available frameworks, cloud computing, and educators such as [andrew-ng](https://www.wikiprompt.org/wiki/andrew-ng), whose online courses introduced the field to a broad audience. It now underpins search ranking, fraud detection, medical imaging analysis, recommendation systems, and the training of large language models that power modern conversational assistants.

The distinction between machine learning and earlier rule-based software is not always sharp in practice, since many production systems combine learned components with hand-coded business logic, but the defining feature of a machine learning system remains that its behavior is shaped substantially by the data it was trained on rather than by rules a programmer wrote directly.

## History

The earliest machine learning program was introduced in the 1950s, when Samuel invented a computer program that calculated the chance of winning in checkers for each side. The history of machine learning is rooted in decades of efforts to study human cognitive processes. In 1949, Canadian psychologist Donald Hebb published *The Organization of Behavior*, in which he introduced a theoretical neural structure formed by certain interactions among nerve cells. The Hebbian theory of neuron interaction set the groundwork for how many machine learning algorithms work, with connected artificial neurons changing the strength of their connections based on data. Other researchers, including Walter Pitts and Warren McCulloch, proposed the first mathematical model of neural networks including algorithms that mirror human thought processes.

By the early 1960s, an experimental "learning machine" with punched tape memory, called Cybertron, had been developed by Raytheon Company to analyse sonar signals, electrocardiograms, and speech patterns using rudimentary reinforcement learning. A representative book on research into machine learning during the 1960s was Nils Nilsson's book *Learning Machines*, dealing mostly with machine learning for pattern classification. In 1981, a report was given on using teaching strategies so that an artificial neural network learns to recognise 40 characters (26 letters, 10 digits, and 4 special symbols) from a computer terminal.

Tom M. Mitchell provided a widely quoted, more formal definition of the algorithms studied in the machine learning field: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E." This definition is fundamentally operational rather than defining the field in cognitive terms. This follows Alan Turing's proposal in his paper "Computing Machinery and Intelligence", in which the question "Can machines think?" is replaced by asking whether machines can convincingly imitate a human in its responses to human-posed questions.

## Relationships to other fields

Machine learning is related to statistics and mathematical optimisation, which provide its foundational methods. Data mining is a related field that focuses on exploratory data analysis through unsupervised learning. From a theoretical viewpoint, probably-approximately-correct-learning provides a mathematical and statistical framework for describing machine learning, and most traditional machine learning and deep learning algorithms can be described as empirical risk minimisation under this framework.

The history of machine learning includes a period when the field shifted away from symbolic approaches towards statistical methods. In the early days of AI, researchers used symbolic methods and early neural networks, but by 1980 expert systems dominated and statistics fell out of favour. Work on symbolic learning continued, but the statistical line of research proceeded in pattern recognition and information retrieval. Neural network research, termed "connectionism", was continued by researchers such as John Hopfield, David Rumelhart, and Geoffrey Hinton, with their main success coming in the mid-1980s with the reinvention of backpropagation. Machine learning as its own field flourished in the 1990s, shifting from achieving artificial intelligence to tackling practical problems with methods from statistics and probability.

## Recent developments

In 2012, AlexNet, developed by [alex-krizhevsky](https://www.wikiprompt.org/wiki/alex-krizhevsky), [ilya-sutskever](https://www.wikiprompt.org/wiki/ilya-sutskever), and [geoffrey-hinton](https://www.wikiprompt.org/wiki/geoffrey-hinton), achieved substantially improved results in the [ImageNet](https://www.wikiprompt.org/wiki/imagenet-dataset) image recognition competition, contributing to the wider adoption of deep neural networks. In 2013, Tomáš Mikolov and colleagues introduced [word2vec](https://www.wikiprompt.org/wiki/word2vec), techniques for efficiently learning distributed vector representations of words from large text corpora. In 2014, [ian-goodfellow](https://www.wikiprompt.org/wiki/ian-goodfellow) and colleagues introduced [generative adversarial networks (GANs)](https://www.wikiprompt.org/wiki/gans), a framework for training generative models through an adversarial process. In 2016, [AlphaGo](https://www.wikiprompt.org/wiki/alphago) became the first computer program to defeat a professional human Go player without handicaps on a full-sized board, using deep neural networks and reinforcement learning. In 2017, [ashish-vaswani](https://www.wikiprompt.org/wiki/ashish-vaswani) and colleagues introduced the [Transformer](https://www.wikiprompt.org/wiki/transformer), a neural network architecture based primarily on [attention](https://www.wikiprompt.org/wiki/attention) rather than recurrence or convolution.

More recently, advances in deep learning have allowed neural networks to surpass many previous machine learning approaches in performance, leading to the terms "machine learning" and "artificial intelligence" being often used interchangeably, even though they are not synonymous.

---
Source: https://www.wikiprompt.org/wiki/machine-learning
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-11-09T12:24:08.79117+00:00