Libratus is an artificial intelligence program designed to play poker, specifically heads-up no-limit Texas hold'em. It was developed at Carnegie Mellon University in Pittsburgh, with its creators intending for it to be generalizable to other, non-poker-specific applications. The name Libratus is a Latin expression meaning 'balanced', and it was the nominal successor of an earlier poker bot, Claudico.
Background and Development
Libratus was built from scratch, but it drew on the legacy of Claudico. Its development required more than 15 million core hours of computation, compared to 2-3 million for Claudico, and was carried out on the 'Bridges' supercomputer at the Pittsburgh Supercomputing Center. According to Professor Tuomas Sandholm, one of Libratus' creators, the bot does not have a fixed built-in strategy; instead, it uses an algorithm that computes the strategy on the fly. The core technique was a new variant of counterfactual regret minimization, namely the CFR+ method introduced in 2014 by Oskari Tammelin. On top of CFR+, Libratus used a novel technique for endgame solving, developed by Sandholm and his PhD student Noam Brown. This method eliminated the need for 'action mapping', which had been the prior de facto standard in poker programming.
Concurrently with Libratus' development, an international team from Charles University, Czech Technical University, and the University of Alberta developed DeepStack, which in December 2016 became the first computer program to defeat professional poker players in heads-up no-limit Texas hold'em. While both systems used counterfactual regret minimization, they differed in their approaches: DeepStack employed neural networks for evaluating game states, whereas Libratus did not use neural networks for leaf evaluation.
2017 Humans versus AI Match
From January 11 to 31, 2017, Libratus competed in a tournament called 'Brains vs. Artificial Intelligence: Upping the Ante challenge' against four top human poker players: Jason Les, Dong Kim, Daniel McAulay, and Jimmy Chou. The tournament involved 120,000 hands, a 50% increase compared to the previous tournament that Claudico played in 2015, and the duration was extended from 13 to 20 days to manage the extra volume.
The four players were grouped into two subteams. One subteam played in the open, while the other subteam was located in a separate room nicknamed 'The Dungeon', where no mobile phones or other external communications were allowed. The Dungeon subteam received the same sequence of cards as the open subteam, but with the sides switched: the Dungeon humans got the cards that the AI got in the open and vice versa. This setup was intended to nullify the effect of card luck.
The prize money of $200,000 was shared exclusively among the human players. Each player received a minimum of $20,000, with the rest distributed based on their performance against the AI. As written in the tournament rules in advance, the AI itself did not receive prize money even though it won the tournament.
During the tournament, Libratus competed against the players during the days, and overnight it perfected its strategy by analyzing the prior day's gameplay, particularly its losses. This allowed it to continuously straighten out imperfections that the human team discovered, resulting in a permanent arms race between the humans and Libratus. The overnight analysis used another 4 million core hours on the Bridges supercomputer.
There was an active betting market among poker players about who would win the tournament. It started with 4:1 odds against the bot, but on the 8th day it became a betting market on which human would lose the least.
Strength of the AI
Libratus led against the human players from day one of the tournament. The player Dong Kim was quoted on the AI's strength as follows: "I didn't realize how good it was until today. I felt like I was playing against someone who was cheating, like it could see my cards. I'm not accusing it of cheating. It was just that good."
On the 16th day of the competition, Libratus broke through the $1,000,000 barrier for the first time. At the end of that day, it was ahead $1,194,402 in chips against the human team. At the end of the competition, Libratus was ahead $1,766,250 in chips and thus won resoundingly. As the big blind in the matches was set to $100, Libratus' winrate is equivalent to 14.7 big blinds per 100 hands, which is considered an exceptionally high winrate in poker and is highly statistically significant.
For comparison, DeepStack achieved a winrate of 49 big blinds per 100 hands in its study against 11 professionals over 44,000 hands, though the different testing conditions (number of hands, opponent skill levels, and tournament format) make direct comparisons difficult.
Of the human players, Dong Kim came first, MacAulay second, Jimmy Chou third, and Jason Les fourth.
The bot has an unconventional betting style where under specific situations it would bet $20,000 even when there's $100 in the pot. After its win, this strategy was analyzed and adopted by the poker community.
Other Possible Applications
While Libratus' first application was to play poker, its designers had a much broader mission in mind for the AI. The investigators designed the AI to be able to learn any game or situation in which incomplete information is available and "opponents" may be hiding information or even engaging in deception. Because of this, Sandholm and his colleagues are proposing to apply the system to other real-world problems as well, including cybersecurity, business negotiations, or medical planning.
See Also
- DeepStack
- Computer poker players
- Cepheus (poker bot)
- Claudico
- Polaris (poker bot)
- Pluribus (poker bot)
References
External Links
- Brains versus Artificial Intelligence Archived 2016-02-03 at the Wayback Machine official website at the Rivers Casino