OpenAI Five is a computer program developed by OpenAI that plays the five-on-five video game Dota 2. It was the first artificial intelligence system to defeat the reigning world champions in a professional esports match, achieving this feat in April 2019. The project aimed to advance machine learning by tackling a game with continuous action spaces, partial observability, and long-term strategy, thereby creating more general problem-solving systems.
The system's development began in November 2016, and its first public appearance was at The International 2017, where it defeated professional player Dendi in a one-on-one match. By 2018, it could play as a full team of five, and in 2019 it won a best-of-three series against OG, the champions of The International 2018. OpenAI Five's algorithms were later adapted for other projects, including a robotic hand. It is often compared to other game-playing AIs like AlphaGo, Deep Blue, and AlphaStar.
History
OpenAI began developing the algorithms for the Dota 2 bots in November 2016. The company chose Dota 2 because it was popular on the streaming platform Twitch, had native Linux support, and offered an application programming interface (API) for developers. The first public demonstration occurred at The International 2017 in August, where the bot defeated Ukrainian professional player Dendi in a live one-on-one matchup. After the match, OpenAI CTO Greg Brockman explained that the bot had learned by playing against itself for two weeks of real time, using reinforcement learning, a method where the system improves by trial and error.
By June 2018, the bots could play together as a full team of five and defeated amateur and semi-professional teams. At The International 2018, OpenAI Five played two exhibition matches against professional teams - paiN Gaming and an all-star team of former Chinese players - but lost both. Despite the losses, OpenAI considered the event successful, as it provided valuable data for refining the algorithms. The final public demonstration was in April 2019, when OpenAI Five won a best-of-three series against OG, the reigning world champions, at a live event in San Francisco. That same month, an online event allowed the public to play against the bots; over four days, the bots played 42,729 games and won 99.4% of them.
Architecture
Each OpenAI Five bot was a neural network with a single layer containing a 4096-unit LSTM (Long Short-Term Memory) that processed the current game state extracted from the Dota 2 API. The network generated actions through multiple action heads, each with a specific meaning, such as the delay before an action, the action type, and the target coordinates. The system observed the world as a list of 20,000 numbers and produced a list of eight values to define an action. It was trained using reinforcement learning on the "Rapid" infrastructure, which managed thousands of machines. By 2018, OpenAI Five had played approximately 180 years of game time, using 256 GPUs and 128,000 CPU cores, with the Proximal Policy Optimization algorithm.
Comparisons with Other Game AI Systems
OpenAI Five differed from earlier game-playing AIs like Deep Blue (chess), AlphaGo (Go), and Watson (Jeopardy!) in several key aspects:
- Long-run view: Dota 2 matches last about 45 minutes at 30 frames per second, resulting in roughly 80,000 ticks per game. OpenAI Five observed every fourth frame, generating 20,000 moves. In contrast, chess typically ends within 40 moves, and Go within 150.
- Partially observed state: Dota 2 features a fog of war that hides enemy units and movements, requiring players to make decisions based on incomplete information. Chess and Go are full-information games with no hidden elements.
- Continuous action space: Each hero in Dota 2 can perform dozens of actions targeting units or positions, with the developers allowing up to 170,000 possible actions per hero. On average, about 1,000 valid actions are available each tick, compared to 35 in chess and 250 in Go.
- Continuous observation space: The game state is represented by 20,000 numbers, whereas a chess board is about 70 values and a Go board about 400.
These characteristics made Dota 2 a more complex challenge for AI, requiring the system to handle real-time decision-making, long-term planning, and teamwork.
Reception
OpenAI Five received widespread recognition from the AI, tech, and gaming communities. Microsoft founder Bill Gates called its victories a "big deal" because they required teamwork and collaboration. Chess champion Garry Kasparov, who lost to Deep Blue in 1997, said that despite the bots' losses at The International 2018, they would "get there, and sooner than expected." AI experts interviewed by MIT Technology Review noted that Dota 2 was an "extremely complicated game," making even victories over non-professional players impressive. PC Gamer wrote that the wins against professional players were a significant milestone for AI research, highlighting the system's ability to handle complex, dynamic environments.