DeepStack

DeepStack is an artificial intelligence computer program designed to play poker, specifically heads up no-limit Texas hold 'em. It is the first computer program to outplay human professionals in this game.

Background

Poker is a key benchmark game in academic community and substantial amount of research was done to find optimal strategies against worst case adversaries. While human professionals have been outplayed in large perfect information games, such as Chess, decades before, imperfect information games require much more complex recursive reasoning. Prior popular approaches relied mainly on simplification of the game by using abstractions. However, abstractions in imperfect-information games often result in highly-exploitable strategies. Instead, DeepStack uses several algorithmic innovations, such as the use of neural networks and continual resolving. The program was developed by an international team from Charles University, Czech Technical University and University of Alberta.

Algorithm

At the core of the program is the use of neural networks for determining the value of specific card combinations. The networks are trained only on a small number of games states and are used to generalize to situations not seen during training. The program uses search with the neural networks and continual resolving to ensure strategy found at each step is consistent with the strategy used in previous steps.

2016 tournament with professional players

In a study completed December 2016 and involving 44,000 hands of poker, DeepStack defeated 11 professional poker players with only one outside the margin of statistical significance. Over all games played, DeepStack won 49 big blinds/100 (always folding would only lose 75 bb/100), over four standard deviations from zero, making it the first computer program to beat professional poker players in heads-up no-limit Texas hold'em poker.