Self-supervised learning is a training approach in which a model learns from data that has no human-provided labels by constructing its own supervisory signal directly from the structure of the data itself. Rather than being told the correct output for each input as in Supervised learning, the model is set a task such as predicting a hidden or missing part of the data from the rest of it, and the "label" for that task is simply the actual value that was hidden, which is already present in the raw, unlabeled dataset at no additional annotation cost.
Core idea and objectives
The canonical example is next-token prediction: given a sequence of text with the final word removed, a model is trained to predict that word, and because ordinary text already contains its own next word, no human ever has to label anything. This is precisely the objective used to pretrain modern large language models, which learn from vast quantities of raw text scraped from sources like Common Crawl purely by repeatedly predicting the next token in a sequence. A related objective, masked-token prediction, hides a random subset of tokens within a sequence and trains the model to fill them back in from the surrounding context; this was the pretraining objective behind BERT. In computer vision and multimodal learning, self-supervised objectives include predicting a masked-out region of an image, and contrastive objectives, used by CLIP, that train a model to recognize which pairs of image and text (or two augmented views of the same image) belong together versus which are mismatched.
Relationship to supervised and unsupervised learning
Self-supervised learning sits conceptually between supervised learning and Unsupervised learning. It works from unlabeled data like unsupervised learning does, but it defines an explicit prediction task with a clear right answer, and it is optimized with the same loss functions, Gradient descent, and Backpropagation machinery used in ordinary supervised training. Some researchers treat it as a subtype of unsupervised learning; others treat it as its own category precisely because its training mechanics resemble supervised learning so closely, differing only in where the labels come from.
Why it matters
Self-supervised learning solved a fundamental bottleneck that had limited supervised learning: the scarcity and cost of human-labeled data. Because the internet and other large corpora already contain enormous amounts of unlabeled text, images, audio, and video, self-supervised objectives allowed models to train on a scale that labeled datasets could never match, a shift that directly enabled the scaling behind the modern foundation model era. It is the mechanism by which Transfer learning became so effective for language: a model pretrained with a self-supervised objective on broad text learns representations general enough to transfer well to a wide range of downstream tasks with comparatively little task-specific Fine-tuning.
Notable milestones and impact
Word2vec's 2013 word-prediction objective is an early influential example of self-supervised learning applied to language, though the term was not yet in wide use at the time. The approach became central to the field's vocabulary and strategy following BERT in 2018 and the GPT series, and by the early 2020s self-supervised pretraining followed by supervised or preference-based fine-tuning had become the standard recipe for essentially every major large language model. The same paradigm extended beyond text: self-supervised video models that predict future frames are a foundational component of research into world models for robotics and autonomous systems. Self-supervised learning's central limitation is that the quality of the resulting representations still depends on the quality, scale, and diversity of the underlying unlabeled data, and biases or gaps present in that data are absorbed into the model just as they would be in supervised training.