Reinforcement learning (RL) is a branch of machine learning in which an agent learns to act within an environment by trial and error, receiving a numerical reward signal after each action and adjusting its behavior to maximize the cumulative reward it collects over time. Unlike Supervised learning, where a model is shown the correct output for each input, reinforcement learning never tells the agent the right action directly; it only reveals, sometimes long after the fact, whether the outcome of a sequence of actions was good or bad, a structure known as the credit assignment problem.
Formal framing and history
Reinforcement learning is typically formalized as a Markov decision process, in which an agent observes a state, chooses an action, receives a reward, and transitions to a new state, repeating this loop while learning a policy that maps states to actions so as to maximize expected future reward. The field's mathematical foundations trace back to dynamic programming and optimal control theory from the mid-20th century, but its modern framing as a distinct machine learning discipline is closely associated with Richard Sutton and Andrew Barto, whose textbook became the field's standard reference, and with researchers who later led DeepMind's reinforcement learning program, including David Silver.
Landmark successes
Reinforcement learning's most publicly visible successes came from games. AlphaGo combined deep neural networks with reinforcement learning and tree search to defeat world champion Lee Sedol at Go in 2016, a result widely seen as arriving years ahead of expert predictions. Its successor AlphaZero generalized the approach, learning chess, shogi, and Go from self-play alone, with no human game data, starting only from the rules. These systems demonstrated that an agent could reach superhuman performance in complex domains purely through reward-driven trial and error combined with enough compute, an idea Sutton later articulated as "the bitter lesson": general methods that leverage computation tend to outperform approaches that rely on hand-engineered human knowledge, as more compute becomes available.
Techniques
Reinforcement learning algorithms fall into several families. Value-based methods, such as Q-learning and its deep-learning extension deep Q-networks, learn to estimate the long-term value of taking a given action in a given state. Policy-gradient methods directly learn a policy function that outputs actions, and actor-critic methods combine both approaches. Proximal Policy Optimization (PPO), developed in large part by John Schulman, became one of the most widely used policy-gradient algorithms due to its relative training stability, and later played a central role outside of games, in the alignment of large language models.
From games to language models
Reinforcement learning's relevance broadened dramatically with RLHF (reinforcement learning from human feedback), in which a reward model trained on human preference judgments substitutes for a hand-coded reward function, and an LLM is then fine-tuned with an algorithm like PPO to produce outputs that score highly according to that learned reward model. This technique was central to turning a raw pretrained Large language model into an assistant like ChatGPT that reliably follows instructions. More recently, reinforcement learning applied directly to tasks with automatically verifiable outcomes, such as math problems with checkable answers or code that either passes or fails tests, became the key technique behind reasoning models such as OpenAI o1 and DeepSeek-R1, which learn to produce longer chains of reasoning that improve their accuracy on hard problems, an approach often described under the umbrella of Test-time compute.
Limitations
Reinforcement learning is notoriously sample-inefficient, often requiring vastly more trial-and-error interactions than a human would need to learn a comparable skill, and it is vulnerable to Reward hacking, where an agent finds an unintended way to maximize its reward signal that technically satisfies the objective while failing the underlying goal it was meant to represent. Designing a reward function that faithfully captures the intended behavior, without creating loopholes an optimizing agent can exploit, remains one of the field's persistent open challenges.