Synthetic data is data that has been artificially generated, typically by a model or simulation, rather than collected from real-world events or human-produced content, and is used to train, fine-tune, or evaluate machine learning systems. In the context of modern large language models, synthetic data usually refers to text, code, or dialogue generated by an existing AI model and then used, often after filtering, as training data for another model, a practice sometimes described as a form of distillation when a stronger model's outputs teach a weaker one.
Synthetic data grew in importance through the mid-2020s as leading labs approached what researchers termed the data wall, a slowdown in the growth of readily available high-quality human-generated text on the internet, making model-generated data an increasingly significant supplement to human-produced text for continued scaling.
Uses
Synthetic data serves several purposes in AI development. It can augment scarce categories of data, such as generating additional examples of rare languages, specialized domains like mathematics or code, or multi-turn conversations that are expensive to collect from humans at scale. It is used extensively in reinforcement learning from human feedback pipelines and their successors, where a model or an ensemble of models generates candidate responses that are then ranked or filtered to create training signal. Synthetic data also plays a central role in training reasoning models, where verifiable domains like mathematics and programming allow a model's own generated solutions to be automatically checked and the correct ones reused as training examples, an approach central to the reinforcement learning methods behind systems such as DeepSeek-R1. Beyond text, synthetic data is widely used in computer vision and robotics, where simulated environments generate labeled images or interaction data that would be costly to collect physically.
The model collapse debate
A body of research beginning around 2023 raised concerns about model collapse, a hypothesized degradation in model quality that could occur if models are trained repeatedly and indiscriminately on data generated by prior models, on the theory that each generation would amplify errors and lose coverage of rare but important patterns from the original human-generated distribution. Subsequent research and industry practice suggested that model collapse is a real risk primarily when synthetic data is used carelessly and without quality filtering or mixing with real data, and that carefully curated or verified synthetic data, particularly in verifiable domains, does not exhibit the same degradation and can substantially improve model quality. By 2025, most frontier labs used substantial quantities of curated synthetic data as a standard part of training, rather than avoiding it.
Quality and verification
Because synthetic data inherits and can amplify the biases, errors, and stylistic quirks of the model that generated it, practitioners generally apply filtering steps, such as using a separate, often more capable, model or a rule-based checker to score or verify quality before including generated examples in a training set. Domains where correctness can be automatically verified, like code that can be executed and mathematics that can be checked, are considered particularly well suited to synthetic data generation because low-quality examples can be filtered out algorithmically, whereas open-ended creative or factual text is harder to verify automatically and carries greater risk of quietly degrading a model trained on it.