poki: the poker agent

Poki: The Poker Agent

Greg PriebeZak Knudson

Overview• Texas Hold’em poker• Architecture and Opponent

Modeling of Poki• Improvements from past Poki• Betting strategies w/ analysis of

hand strengths, weighting, and probabilities.

Texas Hold’em• Each player gets

2 “hole” cards• Community cards:

– 3 “flop” cards– 1 “turn” card– 1 “river” card

• Rounds of betting after each set of cards

Poki Program Architecture

Opponent Modeling• Weight table• No modeling• Generic opponent

modeling• Specific opponent

modeling

The Old Way• Specific opponent modeling• Betting frequency table

– Keeps track of actions taken in different contexts

– Predicts median hand strength• Left out many relevant details

New and Improved• Specific opponent modeling• Detailed• Dynamic learning/adaptation• Prelim. Neural net study

Poki’s Neural Net

Some Results• Training Data• Results from

80-90 percent• Confusion

matrix• Error prone?

In Field Predictions#holdem1: 24% increase

In Field Performance

Small Bets Won

• Pro: .05 - .10• Old opponent

modeling: .09• New opponent

modeling: .22

Betting Strategy• Pre-flop and Post-flop strategies are

significantly different.- Pre: little information available to

influence the betting decision.- Post: uses opponent models, private

hand, and game context to generate an action.

Pre-flop Betting Strategy* - pocket pair hand (two cards of the same rank)s - suited hand o - offsuit hand

Post-flop: Basic Betting Strategy• Compute Poki’s effective hand strength

(EHS).• Using the game context, betting rules,

and formulas to translate the EHS into a probability triple{ Pr(fold), Pr(call), Pr(raise) }.

• Generate a random number and use it to choose an action from the probability distribution.

Hand Strength• Probability that a given hand is better

than that of an active opponent.

Hand Potential• With 2 cards yet to be revealed, we want

to know the impact.• Positive potential (Ppot) – chance hand

improves to win.• Negative potential (Npot) – chance hand

ends up losing.• Calculated by enumerating over all

possible hole cards, and over all possible board cards.

• This is an expensive process.

Hand potential example

Effective Hand Strength• Combines hand strength and

potential to give Poki’s own relative strength against an opponent.

• Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve)

= HS * (1 – Npot) + (1 – HS) * Ppot

Weighting the Enumerations• Probability of hands played to a

particular point will vary.• To account for this, Poki maintains a

weight table for each opponent.

Conclusions• Need opponent modeling • Complex system • Betting strategies and determining hand

strength• Reweight helps to determine

opponents likely action