von neumann poker
TRANSCRIPT
On Von Neumann Pokerwith Community Cards
Reto Spöhel
Joint work with Nicla Bernasconi and Julian Lorenz
Poker
Poker is a popular type of card game in which players gamble on the superior value of the card combination ("hand") in their possession, by placing a bet into a central pot. The winner is the one who holds the hand with the highest value according to an established hand rankings hierarchy, or otherwise the player who remains "in the hand" after all others have folded.
According to Wikipedia,
Outline
Introduction von Neumann Poker von Neumann Poker with community cards Outlook: the Newman model Conclusion
Research on poker
The game of poker has been studied from many different perspectives:
game-theory [this talk] artifical intelligence (heuristics) machine learning (opponent modeling) behavioural psychology etc.
In the game-theoretic approach, one assumes best play for all players involved.
allows development and application of mathematical theories
neglects many other factors
Game-theoretic research on poker
almost exclusively for two-player game two main lines of attack:
simplified models of „real“ poker e.g., 2 suits with 5 cards each solution by brute-force calculation; lots of
computational power needed more abstract models, which hopefully capture
essential features of poker [this talk] e.g., hands are numbers u.a.r. from [0,1] hopefully analytically solvable most important model: von Neumann poker
Von Neumann Poker
P chips are in the pot at the beginning. X and Y are dealt independent hands x,y 2 [0,1] u.a.r. X may make a bet of a or pass („check“).
If X checks, both hands are revealed („showdown“), and the player with the higher hand wins the pot.
If X makes a bet, Y can either match the bet („call“) or concede the pot to X („fold“).
If Y folds, X wins the pot (and gets his bet back). If Y calls X‘s bet, both hands are revealed and the
player with the higher hand wins the pot and the two bets.
In the following we assume for simplicity P = a = 1.
Von Neumann Poker
It seems that X has an advantage, since Y can only react. So how should X play to maximize his expected payoff?
Clearly, always checking guarantees him an expected payoff of P/2 = 1/2.
Similarly, X cannot hope for an expected payoff of more than P=1, since Y can always fold.
Von Neumann Poker
At first sight, one might guess that X should bet the better half of his hands, i.e., iff x ¸ 1/2.
However, once Y realizes that this is X‘s strategy, he will only call with hands y ¸ 2/3, since then
he wins P+a=2 chips with probability at least 1/3 he loses a=1 chips with probability at most 2/3 i.e., „the pot odds are in his favor“.
2/3
y
1/2
0
1
x
check
bet
fold
call
Von Neumann Poker
X‘s expected payoff can be found by integrating over the hands of x and y and is:
P ¢ 1/8 + P ¢ 1/2 ¢ 2/3 + (P+a) ¢ 1/18 – a ¢ 1/9
= 1/8+1/3 = 11/24 < 1/2 X loses money!
check
0
bet-call
P
bet-fold
P
-a
P+a2/3
0x
y
1/2
1/2
2/3
0
1
check
bet
fold
call
x y
So X has no advantage over Y and should never bet? NO!
With the previous strategy, most of X‘s good hands go to waste because Y just folds.
However, X can induce Y to call more often by including bluffs in his strategy!
von Neumann gave an equilibrium pair of strategies Y‘s strategy is best response to X‘s strategy X‘s strategy is best response to Y‘s strategy
X can achieve an expected payoff of 5/9, which is optimal.
von Neumann, 1928
Von Neumann Poker
von Neumann‘s solution
Heuristic: we make the following ansatz: With a hand of y1, calling and folding should have the
same expected payoff for Y:
x0 ¢ (P + a) – (1 – x1) ¢ a = 0 With a hand of x0 or x1, betting and checking should
have the same expected payoff for X:
y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – y1) ¢ a = x0 ¢ P
y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – x1) ¢ a = x1 ¢ P
0
1
check
value-bet
fold
call
x y
bluff-bet
y1
x1
x0
solution (P=a=1): x0 = 1/9 y1 = 5/9 x1 = 7/9
von Neumann‘s solution
The two resulting strategies are indeed in equilibrium. The expected payoff for X turns out to be 5/9
the value of the game is 5/9 (in zero-sum games, all equilibria have the same value!)
Insights: Bluffing is a game-theoretic necessity! You should bluff-bet your worst hands!
solution (P=a=1): x0 = 1/9 y1 = 5/9 x1 = 7/90
1
check
value-bet
fold
call
x y
bluff-bet
y1
x1
x0
Von Neumann Poker
Many extensions of the von Neumann model have been studied
allow multiple betting rounds, raises, reraises, etc. hands may depend on each other… etc.
by scientists and professional poker player alike.
Chris Ferguson, PhD 2000 World Series
of Poker champion co-author of
several papers on von Neumann poker
The mathematics of Poker, Bill Chen and Jerrod Ankenman, 2006
Our contribution
In the classical von Neumann model (and its extensions), no further random influences are present once both players have received their hands.
In real poker, community cards are drawn between betting rounds.
players may bet with a bad hand, hoping that the right card will show up and turn it into a good hand.
players with a good hand tend to bet more aggressively to force other players to fold.
We propose the following extension of the von Neumann model that accounts for these features:
Before the showdown, throw an unfair coin. With probability q, the lower hand wins!
Introducing the flip
Introducing the flip: With a hand of y1, calling and folding should have the
same expected payoff for Y: before (q=0):
x0 ¢ (P + a) – (1 – x1) ¢ a = 0 now:
[x0 ¢ (1 – q) + (1 – x1) ¢ q] ¢ (P + a)
– [(1 – x1) ¢ (1 – q) + x0 ¢ q] ¢ a = 0
solution (P=a=1): x0 = x0(q) y1 = y1(q) x1 = x1(q)
0
1
check
value-bet
fold
call
x y
bluff-bet
y1
x1
x0
Introducing the flip
y1
x1
x0
?
1/3 =: q0
5/9
1/9
7/9
We obtain
Beyond the critical q
What happens for q > q0 = 1/3? Y will call every bet since even with the worst hand y = 0
he wins P+a=2 chips with probability at least q = 1/3 he loses a=1 chips with probability at most 1 – q = 2/3
Knowing this, X will bet the better half of his hands.
1/2
0
1
check
bet
call
x y
for general P and a:
q : (1-q) = a : (P+a)
q0 = a/(P+2a)
The full picture
y1
x1
x0
q0 = 1/3
As q increases, X makes more value bets and bluffs less.
Y is induced by the q to call! For q ¸ q0, there‘s no point in
bluffing, since Y will always call anyway.
X value-bets more often to protect his good hands from being flipped into bad hands.
The value of the game
1/2 = 0.5
7/12 = 0.583…
5/9 = 0.555…
q0 = 1/3
value of the game
The expected payoff of X is maximal at q = q0.
i.e., when Y has just enough incentive to call every bet of X without X wasting money on bluffs.
An improved model
Where does the discontinuity at q0 come from? We fixed the bet size a (arbitrarily) before the game
started. What if we allow X to look at his hand and then bet any
amount a ¸ 0 he likes?
…and with flip probability q, 0 · q < 1/2
X can achieve an expected payoff of 4/7, which is optimal.
Newman, 1959
X can achieve an expected payoff of (16-q)/(28-8q), which is optimal.
Bernasconi, Lorenz, S., 2007+
Newman‘s solution
1/7 4/7
„value-bet“„bluff-bet“
bet a
hand x, resp. y
The red line is X‘s bet a(x) X checks for 1/7 · x · 4/7
The green line is Y‘s calling threshold
Y calls iff (y,a) is below the green line
obtained with similar, but more involved heuristics than before( differential equations)
(q = 0)
Introducing the flip
a0 =
1/2
Similarly to before:
q : (1-q) = a : (P+a)
a0 = q/(1-2q) ¢ P even with his worst hand, Y calls
every bet of at most a0 (pot-odds!) knowing this, X never bets an
amount between 0 and a0
neither as a value bet (if the odds are in his favour [x ¸ 1/2], he bets at least a0),
nor as a bluff bet (there‘s no value bet for which to induce more calls).
hand x, resp. y
(q = 1/3)
bet a
Introducing the flip
(q = 1/3)
(q = 0.4)
bet a
As q increases, X bets his value bets more aggressively.
hand x
The value of the game
value of the game
For 0 < q · 1/2, we have
value(q) = (16-q)/(28-8q)
the value is strictly increasing in q but discontinuous at q=1/2, since trivially
value(1/2) = 1/2.
What is going on?
We allowed arbitrarily high bets a. Moreover, a0 diverges at q = 1/2.
If we limit X‘s bankroll, the singularities vanish.
4/7 = 0.571…
31/48 = 0.645…
Conclusion
We proposed a way of including „community cards“ (random effects after betting) into both the von Neumann and the Newman model…
… and gave complete analytical solutions. As expected, we observed increasingly aggressive betting
for larger q. In both models, we observe a „phase transition“ when
q : (1-q) = a : (P+a) (“flip-odds = pot-odds”)
Questions?