Game Theory and Poker

[sjb]

Good show. A lot of weird "game theory" claims do show up. In support of the section "Fallacious Concept 4: An Equlibrium Strategy is a Good Strategy", let me point out that, in all forms of poker I know of (certainly HE, Omaha, and Stud, and regardless of limit structure or high/low splits) and in the absence of a rake, the EV of any Nash equilibrium strategy is exactly zero. With a rake, the EV is negative and equal to the total amount of the rake, divided by the number of players.

In any given hand, the EV would vary by position. In HE the highest EV would be on the button and decreases in earlier positions. But since each player eventually sits in each seat an equal number of times, those all average out.

The EV of the equilibrium strategy is the expectation when everyone else is playing their corresponding strategy in the equilibrium. If all players are playing optimally, then a non-zero EV comes from asymmetries in the game. In poker, that asymmetry is position. But since the button rotates, as you go from a single hand to repeated hands, that asymmetry is destroyed and the result is that everyone breaks even.

Maybe the most problematic thing about using game theory in poker is that game theory doesn't tell you how to win. Because game theory assumes all players are "rational" (that is, they play so as to maximize utility - in poker that means they play to make the most money possible) and "intelligent" (that is, they're at least as capable at analyzing the game as we are), game theory doesn't tell you how to win. It tells you how to not lose.

The EV of an equilibrium strategy represents the worst you could do. The best you could do depends on exactly how your opponents deviate from their equilibrium strategies. Game theory would still tell you how to analyze that game, but the opponent's strategy becomes part of the game and it has to be re-analyzed for each new set of opponents.

So, while game theory can make interesting contributions to poker theory, it's hardly the last word.

[Chipp Leider]

Actually I think the main problem is people thinking GT is somehow restricted to just optimal play, which is not at all the case.

[mustlearnpoker]

what is ev

[jogsxyz]

"Players are assumed to be rational:"

How did Lunsford come up with that? If players were rational, why bother to play? We would all lose the rake.
We hope opponents play badly, but predictably badly. And we hope to exploit their bad tendencies.

[JaredL]

Thank you for the feedback, I'm glad you liked the article.

You said:
[ QUOTE ]
in all forms of poker I know of (certainly HE, Omaha, and Stud, and regardless of limit structure or high/low splits) and in the absence of a rake, the EV of any Nash equilibrium strategy is exactly zero. With a rake, the EV is negative and equal to the total amount of the rake, divided by the number of players.

[/ QUOTE ]

This isn't quite true. The EV (Expected Value, or average earn) of the Nash Equilibrium will be zero minus rake, but the EV of the NE strategies won't necesarily be. If the other players aren't playing their NE strategy then it could be different. In the bluffing example the bluffers EV doesn't depend on the other guy's strategy but in other cases it might. For example, the caller in the bluffing situation should call or fold with the right frequency to make the bettor indifferent between bluffing and checking. If the bettor changes her strategy to, for example, always betting then the caller is no longer breaking even with the Nash strategy and will have won a positive amount in expectation.

The paragraph about GT showing how not to lose versus how to win is good, as is the bit about opponents' strategies. The next edition will discuss how to use your opponents' strategies to put them on a hand range.

Thanks again for the feedback.

[JaredL]

[ QUOTE ]
Actually I think the main problem is people thinking GT is somehow restricted to just optimal play, which is not at all the case.

[/ QUOTE ]

GT results are based on the assumption that the players in the game are playing, in some sense, optimally. So it is restricted to optimal play.

If you're pointing out that you can use the same techniques to analyze games where players aren't using optimal strategies then we certainly agree, and that is the main reasoning behind the article series.

[JaredL]

[ QUOTE ]
"Players are assumed to be rational:"

How did Lunsford come up with that? If players were rational, why bother to play? We would all lose the rake.
We hope opponents play badly, but predictably badly. And we hope to exploit their bad tendencies.

[/ QUOTE ]

You are exactly right!

My point is that game theory results rely on the assumption that players are rational. This makes it much easier to analyze strategic interactions than using other assumptions for players' behavior and motivations. This, however, makes game theory results fairly worthless for helping you figure out the best move in an actual game against actual people.

In poker other players are far from rational. The clearest example of this is a player calling on 7th street in Stud without being able to beat the other players board. A point I make in the article, and this will be expanded upon in the future, is that you can still use some of the techniques from game theory to help you determine the best play even if your opponents' are not playing optimally.

Jared

[StellarWind]

[ QUOTE ]
... game theory doesn't tell you how to win. It tells you how to not lose.

The EV of an equilibrium strategy represents the worst you could do. The best you could do depends on exactly how your opponents deviate from their equilibrium strategies.

[/ QUOTE ]
Actually an equilibrium strategy doesn't even accomplish the minimal goal of not losing when there are more than two players. If everyone follows the equilibrium strategy than the EV of each player across an entire orbit will be zero and you won't lose. But one player not following the equilibrium strategy can easily place another "innocent" player in an unavoidable negative EV situation.

There are many practical examples of this in poker. Suppose someone decides that good LHE strategy involves always raising/reraising preflop from the Button. You don't want to sit two seats to his left because he'll raise everytime you post the BB. Sure he's spewing chips but the whole table benefits from that and not just you. It doesn't make up for the fact that your BB equity is being redistributed to all the other players at the table.

Another problematic player is the small blind on your immediate right who always bets the flop into five people. Sure he's full of it but that doesn't mean you can put money in with four people behind you. You wind up folding a lot of hands you might have won.

Then there's the action player on your immediate left who straddles your big blind every orbit. He's killing your blind equity and selectively raising the effective stakes on deals where you are out-of-position.

The final example is collusion where your opponents are helping each other but not communicating with each other. This type of play does not violate the article's mathematical assumption of a noncooperative game because each player is acting independently. The player's intentions may be bad but his plays are just another possible strategy. This is why people can be concerned about a husband and wife playing together even if they are absolutely certain they are not signaling hole cards or betting plans. In fact it only takes one spouse to cheat and the other may not even realize what is happening.

[jogsxyz]

JaredL,

I just noticed that you are the mod for this section. Are you
familiar with the 3-card deck game. It has been discussed on
2+2. I lack the ability to navigate the 2+2 forum. Don't know
if the solutions are posted within the forum. I have posted
the solution to the game on my blog. Both the special case and
the general case are detailed in the blog.

http://jogsxyz.blogspot.com/

jogs

[jogsxyz]

[ QUOTE ]

Fallacious Concept 1: Game theory is very useful for poker players

Strange that I would write this in a poker article about game
theory, but this is a misnomer. Other than a few select examples
such as bluffing described in The Theory of Poker, game theory is
generally not practically useful. Even these examples are of
little use (see Fallacious Concept 4 below). One of the reasons
for this I alluded to above. The assumption that agents are
rational is clearly not valid when you are playing poker. If that
were the case the Noted Poker Authority would certainly have far
less to write about. Players seem to bet, raise, and call for no
good reason all the time.

[/ QUOTE ]


Most of the players are semi-rational. It's just that they may not
be on the same wave length as us on strategy. Are they consistent
and predictable? To some degree they are. At least we can make
assumptions which should produce better results than assuming
all actions by opponents are random.
On earlier streets, streets before the river it's less game
theory doesn't apply than with our limited understanding of game
theory the simplistic models are not very useful for most game
situations. In the future we should be able to construct better
models to solve more game situations. But game theory will never
be able to quantify and assign proper values to opponents'
tendencies. That will be an exercise for each player.

[JaredL]

[ QUOTE ]

I just noticed that you are the mod for this section. Are you
familiar with the 3-card deck game.

[/ QUOTE ]

I looked at what you wrote, seems interesting. I didn't thoroughly read it, probably will some time next week. As a homework exercise in a micro class we had to solve the 3 card game where the opponent has middle and the other player has high or low. The hi/lo guy can check for a showdown or bet. If he bets then the middle guy can call or fold. IIRC we did the version where the probability of having a high hand is p and low is 1-p, not just .5/.5.

Jared

[JaredL]

[ QUOTE ]

Most of the players are semi-rational. It's just that they may not
be on the same wave length as us on strategy. Are they consistent
and predictable? To some degree they are. At least we can make
assumptions which should produce better results than assuming
all actions by opponents are random.


[/ QUOTE ]

When I say players are not rational, I'm not saying that they play randomly just that they don't follow the game theoretic definition of rationality that is assumed.

[ QUOTE ]

On earlier streets, streets before the river it's less game
theory doesn't apply than with our limited understanding of game
theory the simplistic models are not very useful for most game
situations. In the future we should be able to construct better
models to solve more game situations. But game theory will never
be able to quantify and assign proper values to opponents'
tendencies. That will be an exercise for each player.

[/ QUOTE ]

Not sure what you mean in the first sentence. As for the end, just to promote my next article, I will be discussing specifically how "to quantify and assign proper values to opponents' tendencies." This is an exercise for the player, but the best way is to use techniques from game theory.

Jared

[jogsxyz]

[ QUOTE ]
[ QUOTE ]

I just noticed that you are the mod for this section. Are you
familiar with the 3-card deck game.

[/ QUOTE ]

I looked at what you wrote, seems interesting. I didn't thoroughly read it, probably will some time next week. As a homework exercise in a micro class we had to solve the 3 card game where the opponent has middle and the other player has high or low. The hi/lo guy can check for a showdown or bet. If he bets then the middle guy can call or fold. IIRC we did the version where the probability of having a high hand is p and low is 1-p, not just .5/.5.

Jared

[/ QUOTE ]

I've also posted the solution for the general case. You may vary p, the bet size and the pot size.

[Whirly]

Just a minor whinge. I am not sure that the description of Nash Equilibrium is wrong in "A Beautiful Mind"? Isn't Nash supposing that if anyone were to alter their strategy and go for the Blonde then that would cause the others to similarly alter their strategy? Thus they are optimal if they stick with their own brunette! It's a kind of Prisoner's Dilemma situation.

/images/graemlins/confused.gif

Whirly
Q. If I've got Aces, should I tell you to fold or raise?
A. Neither, just get me another beer and play the damn game.

[StellarWind]

[ QUOTE ]
Just a minor whinge. I am not sure that the description of Nash Equilibrium is wrong in "A Beautiful Mind"? Isn't Nash supposing that if anyone were to alter their strategy and go for the Blonde then that would cause the others to similarly alter their strategy? Thus they are optimal if they stick with their own brunette! It's a kind of Prisoner's Dilemma situation.

[/ QUOTE ]
As I understand the article, in a true Nash Equilibrium going after the blonde would not improve your results even if no one else altered their strategy. Obviously that isn't true if everyone else is pursuing a strategy of only courting brunettes.

Returning to poker, in an equilibrium situation bluffing more than the equilibrium amount cannot make you more money even if no one else compensates by calling more.

[JaredL]

[ QUOTE ]
Just a minor whinge. I am not sure that the description of Nash Equilibrium is wrong in "A Beautiful Mind"? Isn't Nash supposing that if anyone were to alter their strategy and go for the Blonde then that would cause the others to similarly alter their strategy? Thus they are optimal if they stick with their own brunette! It's a kind of Prisoner's Dilemma situation.

[/ QUOTE ]

A Nash equilibrium is a situation where given what your opponents are doing, you are acting optimally (and that's true for everyone). It doesn't matter how your opponents would respond to you deviating, they simply assume that you are doing whatever it is that you're doing. So in the example if any of the guys recognizes that the others are going after brunettes he should go after the blonde (I personally prefer the darker haired ladies but the guys in the movie like the blondes). It's possible, as you suggest, that the scenario where one individual guy is going after the blonde and the rest are after the brunettes is not a NE but that also doesn't affect the conclusion that all going after the brunettes is not a NE.

[sjb]

[ QUOTE ]
Thank you for the feedback, I'm glad you liked the article.

You said:
[ QUOTE ]
in all forms of poker I know of (certainly HE, Omaha, and Stud, and regardless of limit structure or high/low splits) and in the absence of a rake, the EV of any Nash equilibrium strategy is exactly zero. With a rake, the EV is negative and equal to the total amount of the rake, divided by the number of players.

[/ QUOTE ]

This isn't quite true. The EV (Expected Value, or average earn) of the Nash Equilibrium will be zero minus rake, but the EV of the NE strategies won't necesarily be. If the other players aren't playing their NE strategy then it could be different. In the bluffing example the bluffers EV doesn't depend on the other guy's strategy but in other cases it might. For example, the caller in the bluffing situation should call or fold with the right frequency to make the bettor indifferent between bluffing and checking. If the bettor changes her strategy to, for example, always betting then the caller is no longer breaking even with the Nash strategy and will have won a positive amount in expectation.

[/ QUOTE ]

Maybe I didn't state it clearly, but when I was talking about the EV of the equilibrium strategy being zero (less the rake), I was talking about the game-theoretic "value" of the game - when players deviate from the strategy dictated by the equilibrium, then of course it's possible (even likely) for the EV to go up. But then you're no longer talking about the EV of the equilibrium strategy - you're talking about the EV of some non-equilibrium. You're not in equilibrium unless both players are playing that way.

[sjb]

[ QUOTE ]
In poker other players are far from rational. The clearest example of this is a player calling on 7th street in Stud without being able to beat the other players board. A point I make in the article, and this will be expanded upon in the future, is that you can still use some of the techniques from game theory to help you determine the best play even if your opponents' are not playing optimally.

[/ QUOTE ]

Actually, game theory is about studying the strategies when all players are rational. If you're going to study scenarios where only one of the players is "rational", then you have to build a model of the supposed "irrational" behavior of the other players. Once you've done that, you don't need game theory anymore - you just need the (earlier) discipline that game theory was built on: decision theory.

Very much of the poker literature that discusses game theory is really discussing decision theory. When you see "assume he would raise with any high pair and call with any suited ace" or similar hypothetical scenarios, you've built a model of the other player, who's no longer a "player" in the game-theoretic sense, but now just a stochastic process that you've statistically modelled.

[jogsxyz]

There's more than one strategy available to the players. There's optimal strategy and exploitive strategy.

Let's say Bob plays a fixed strategy that's not optimal. Now Ann can adjust her strategy to exploit Bob's tendencies. This is still part of game theory.

Game theory is not a fixed strategy. Optimal strategy is a subset of game theory.

Bob plays a poor strategy. Ann can play an adjusted strategy to maximize on Bob's poor play. Then Ann can have +EV. The sum of Ann's EV plus Bob's EV plus the rake will still equal zero.

[JaredL]

[ QUOTE ]
[ QUOTE ]
Thank you for the feedback, I'm glad you liked the article.

You said:
[ QUOTE ]
in all forms of poker I know of (certainly HE, Omaha, and Stud, and regardless of limit structure or high/low splits) and in the absence of a rake, the EV of any Nash equilibrium strategy is exactly zero. With a rake, the EV is negative and equal to the total amount of the rake, divided by the number of players.

[/ QUOTE ]

This isn't quite true. The EV (Expected Value, or average earn) of the Nash Equilibrium will be zero minus rake, but the EV of the NE strategies won't necesarily be. If the other players aren't playing their NE strategy then it could be different. In the bluffing example the bluffers EV doesn't depend on the other guy's strategy but in other cases it might. For example, the caller in the bluffing situation should call or fold with the right frequency to make the bettor indifferent between bluffing and checking. If the bettor changes her strategy to, for example, always betting then the caller is no longer breaking even with the Nash strategy and will have won a positive amount in expectation.

[/ QUOTE ]

Maybe I didn't state it clearly, but when I was talking about the EV of the equilibrium strategy being zero (less the rake), I was talking about the game-theoretic "value" of the game - when players deviate from the strategy dictated by the equilibrium, then of course it's possible (even likely) for the EV to go up. But then you're no longer talking about the EV of the equilibrium strategy - you're talking about the EV of some non-equilibrium. You're not in equilibrium unless both players are playing that way.

[/ QUOTE ]

We're basically just arguing semantics. As I said, I think you're right you're just using the wrong terminology.

A strategy is a plan of action for all of the various scenarios that could come up in the game. A particular strategy is a/the (Nash) equilibrium. The EV of any strategy (including the NE strategy) depends on the strategy the other players are using. Hence the EV of the NE strategy will vary depending on what the other players are doing. So the game theoretic "value" of the game is the EV of the equilibrium strategy when the opponent plays her equilibrium strategy. There is nothing special about an equilibrium strategy other than that it leads to an equilibrium if the other players act accordingly. You can still look at its EV even if they don't.

Again, I think we're just splitting hairs on the definition of 'equilibrium' and 'equilibrium strategy'.

[JaredL]

[ QUOTE ]
[ QUOTE ]
In poker other players are far from rational. The clearest example of this is a player calling on 7th street in Stud without being able to beat the other players board. A point I make in the article, and this will be expanded upon in the future, is that you can still use some of the techniques from game theory to help you determine the best play even if your opponents' are not playing optimally.

[/ QUOTE ]

Actually, game theory is about studying the strategies when all players are rational. If you're going to study scenarios where only one of the players is "rational", then you have to build a model of the supposed "irrational" behavior of the other players. Once you've done that, you don't need game theory anymore - you just need the (earlier) discipline that game theory was built on: decision theory.

Very much of the poker literature that discusses game theory is really discussing decision theory. When you see "assume he would raise with any high pair and call with any suited ace" or similar hypothetical scenarios, you've built a model of the other player, who's no longer a "player" in the game-theoretic sense, but now just a stochastic process that you've statistically modelled.

[/ QUOTE ]

There are equilibrium refinements that rely on players assuming that other players are rational. One of them involves forward induction, which conveniently is discussed in my next article currently up. For the standard Nash Equilibrium that's not the case. When you show that whatever strategy n-tuple you've found is an equilibrium you start by assuming that players 1 through n-1 play their equilibrium strategy and show that a best response for n is her equilibrium strategy. When you assume that 1-(n-1) are playing their equilibrium strategy, you don't assume anything about rationality, just that they play their equilibrium strategy. Rationality comes into play only when assessing whether or not n's strategy is a best response.

It's like that in poker as well. You assume that your opponents are playing some strategy and then if you are Ray Zee you will always play a best response to what they are playing. In solving games you put yourself in player n's shoes looking at what the other players are doing and respond accordingly. In poker you actually are that player n. The only difference is that in poker the opponents aren't rational and won't be playing a best response to what you're doing. That doesn't affect the analysis.

You are correct that it's not technically a game theory problem because our opponents aren't rational, but we don't care about them because we are (attempting to be) rational. The thought process is identical.

edited part in bold

edit #2: just want to clarify from the first paragraph that eventually you will need to assume that players 1 through n-1 are rational but only one at a time when you are showing that their NE strategy is a best response to the other players' strategies. So you basically assume one at a time that they are rational. In poker we're always trying to be the rational one which is why the thinking is so similar.

[sjb]

[ QUOTE ]
[ QUOTE ]
... game theory doesn't tell you how to win. It tells you how to not lose.

[/ QUOTE ]

Actually an equilibrium strategy doesn't even accomplish the minimal goal of not losing when there are more than two players. If everyone follows the equilibrium strategy than the EV of each player across an entire orbit will be zero and you won't lose. But one player not following the equilibrium strategy can easily place another "innocent" player in an unavoidable negative EV situation.

[/ QUOTE ]

Yes, and no. If player A departs from the equilibrium strategy in such a way that an "innocent" player (B) is in a "negative EV situation", then by definition, B has a different strategy they could follow that would be better than the equilibrium strategy they were originally following, and so can obtain a non-negative EV - this shifts the negative EV to some other "innocent", who can also switch to a better strategy, and so on.

Ultimately, all of the players can (in theory) switch strategies so that the negative EV falls back on the player who's playing a non-equilibrium strategy. The other players may not be able to guarantee a positive EV in their new equilibrium, but they can at least remain non-negative.

[StellarWind]

[ QUOTE ]
Yes, and no. If player A departs from the equilibrium strategy in such a way that an "innocent" player (B) is in a "negative EV situation", then by definition, B has a different strategy they could follow that would be better than the equilibrium strategy they were originally following, and so can obtain a non-negative EV - this shifts the negative EV to some other "innocent", who can also switch to a better strategy, and so on.

Ultimately, all of the players can (in theory) switch strategies so that the negative EV falls back on the player who's playing a non-equilibrium strategy. The other players may not be able to guarantee a positive EV in their new equilibrium, but they can at least remain non-negative.

[/ QUOTE ]
This is not correct. If player A deviates from the equilibrium strategy in a multiplayer poker game then player B may be screwed. He may have no strategy that avoids negative EV. Players A and B both lose and the other players gain.

I believe I gave some valid poker examples of this in my last post, but poker is complicated and proof can be elusive.

Here is a non-poker example that illustrates my point. Three players play a game in which each player antes $1. The first player chooses a whole number between one and three and annouces it out loud. Then the second player chooses a number between one and three and announces it out loud. Finally the last player chooses his number. Choosing duplicate numbers is allowed.

Now a "3-sided die" is rolled to generate a random number between 1 and 3. Each player who chose that number gets an equal share of the pot. If no one picked the number the game is a chop and the antes are returned.

Clearly it is bad to have the same number as someone else. Optimal strategy for each player is to pick a number that no one else has picked yet. This is also the Nash Equilibrium strategy for all players and it yields EV=0 for each player if everyone follows it.

I'm player "A". Whenever I am second to act my strategy is to choose the same number announced by player B sitting on my right. This is a really bad strategy and it is going to cost me a lot of money. But notice that Player B has the same EV as I do. He's played perfectly but he is no better off than I am and there is nothing he can do to improve his situation. We're both losing money to player C on my left.

[jogsxyz]

http://www.cardplayer.com/poker_magazine/archives/?a_id=14980&m_id=65572

sjb, read Matt Matros' article on game theory.