poki: the poker agent
DESCRIPTION
Poki: The Poker Agent. Greg Priebe Zak Knudson. Overview. Texas Hold’em poker Architecture and Opponent Modeling of Poki Improvements from past Poki Betting strategies w/ analysis of hand strengths, weighting, and probabilities. Texas Hold’em. Each player gets 2 “hole” cards - PowerPoint PPT PresentationTRANSCRIPT
Poki: The Poker Agent
Greg PriebeZak Knudson
Overview• Texas Hold’em poker• Architecture and Opponent
Modeling of Poki• Improvements from past Poki• Betting strategies w/ analysis of
hand strengths, weighting, and probabilities.
Texas Hold’em• Each player gets
2 “hole” cards• Community cards:
– 3 “flop” cards– 1 “turn” card– 1 “river” card
• Rounds of betting after each set of cards
Poki Program Architecture
Opponent Modeling• Weight table• No modeling• Generic opponent
modeling• Specific opponent
modeling
The Old Way• Specific opponent modeling• Betting frequency table
– Keeps track of actions taken in different contexts
– Predicts median hand strength• Left out many relevant details
New and Improved• Specific opponent modeling• Detailed• Dynamic learning/adaptation• Prelim. Neural net study
Poki’s Neural Net
Some Results• Training Data• Results from
80-90 percent• Confusion
matrix• Error prone?
In Field Predictions#holdem1: 24% increase
In Field Performance
Small Bets Won
• Pro: .05 - .10• Old opponent
modeling: .09• New opponent
modeling: .22
Betting Strategy• Pre-flop and Post-flop strategies are
significantly different.- Pre: little information available to
influence the betting decision.- Post: uses opponent models, private
hand, and game context to generate an action.
Pre-flop Betting Strategy* - pocket pair hand (two cards of the same rank)s - suited hand o - offsuit hand
Post-flop: Basic Betting Strategy• Compute Poki’s effective hand strength
(EHS).• Using the game context, betting rules,
and formulas to translate the EHS into a probability triple{ Pr(fold), Pr(call), Pr(raise) }.
• Generate a random number and use it to choose an action from the probability distribution.
Hand Strength• Probability that a given hand is better
than that of an active opponent.
Hand Potential• With 2 cards yet to be revealed, we want
to know the impact.• Positive potential (Ppot) – chance hand
improves to win.• Negative potential (Npot) – chance hand
ends up losing.• Calculated by enumerating over all
possible hole cards, and over all possible board cards.
• This is an expensive process.
Hand potential example
Effective Hand Strength• Combines hand strength and
potential to give Poki’s own relative strength against an opponent.
• Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve)
= HS * (1 – Npot) + (1 – HS) * Ppot
Weighting the Enumerations• Probability of hands played to a
particular point will vary.• To account for this, Poki maintains a
weight table for each opponent.
Conclusions• Need opponent modeling • Complex system • Betting strategies and determining hand
strength• Reweight helps to determine
opponents likely action