Fictitious play for multi-player games

[trojanrabbit]

I think the difference lies in the definition of the "game." Fictitious play will work (I've used it) if you assume the current hand is a one-shot deal. There are no more interactions after the current hand. However if you extend the definition of the game to cover multiple hands then it gets a lot more complicated.

A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

Tysen

[jukofyork]

[ QUOTE ]
I think the difference lies in the definition of the "game." Fictitious play will work (I've used it) if you assume the current hand is a one-shot deal. There are no more interactions after the current hand. However if you extend the definition of the game to cover multiple hands then it gets a lot more complicated.

A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

[/ QUOTE ]
Yep, I guess this would require expanding the game tree out to be able to see the blinds moving and the big stack getting into more and more +EV bullying situations. Perhaps it could be expanded into the next hand (or even next few hands) and still be computationally tractable? Not sure how much better the solutions would be though.

Juk [img]/images/graemlins/smile.gif[/img]

[plexiq]

Kind of forgot about this thread, sorry [img]/images/graemlins/smile.gif[/img]

[ QUOTE ]
A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

Tysen

[/ QUOTE ]

I think the example is actually mixing in a different problem.

One part of the problem you describe boils down to flaws of ICM. ICM overestimates midstack-equities at the bubble, and underestimates bigstack equity. If we had access to a better EQ-estimation, midstacks would automatically call wider, because relative equities of folding/busting/doubling up would change.

With ICM we have lots of scenarios where players are expected to win/lose equity during the next orbit. This should never be the case with an accurate EQ model.

As i understand it, thats to be seen "separated" from our original problem: That the NE is usually a bad state for the caller, because he is actually in the position to "force" the pusher into a more favorable state. I think this is a problem with the NE altogether. Maybe i can think of some toy game to better demonstrate my though,...(hopefully i wont forget about the thread, again [img]/images/graemlins/laugh.gif[/img])

[plexiq]

Ok, here is a toy game featuring a "spite-calling" situation:

Basic game is the same as in Math of Poker, pg 127.

*) Every player is dealt a hand in [0...1].
*) SB ("Pusher") can push or fold
*) If SB pushes, BB ("Caller") can call or fold.
*) If there is a showdown, the player with the higher hand has 2/3 pot equity.

We use stacks of 5BB (SB=0.5, BB=1).

So far, thats just "normal" HeadsUp - and there s no possible spite-calling. After all, we are still in a zero-sum game atm. The NE for this "base game" is: Pusher: 70%, Caller: 56%.

Now lets add the possibility to "spite call":
We now change the game, such that the players will convert their stacks to money after the game, and the players goal is to optimize their $EV. However, the conversion is non-linear. Their stack will be converted to money by payout(chips)=sqrt(chips). (Any strictly growing function will do, as long as it grows "slower than linear". Sqrt is an arbitrary choice.)

This models to some degree the situation of an SNG, because doubling up in chips will now be worth less than double $.

In this modified game, the NE would be:
Pusher: Top 100%, Caller: 8.6%.

We are only 5BB deep, and NE suggests that BB is only calling 8.6% against an ATC push. Alright so far [img]/images/graemlins/smile.gif[/img]



In the plot you can see that the Caller can deal huge "EV-damage" to the pusher, by sacrificing very little EV himself. I think that the NE is unsuitable in this situation, because the caller could clearly "force" the pusher into a more favorable state.

[Paxinor]

to simulate a sit n go properly, wouldn't it be suitable to create a game where the sum of $EV is always the same? i mean this is the crucial point, because it needs to be a zero sum game! and ICM is a zero sum game too...

[plexiq]

What we want to simulate here, is the SB-vs-BB "subgame", after n other players folded. ICM isnt zero sum in this situation (if we only look at the involved players).

[Paxinor]

ah ok know i get it...

well this is why multiplayer games suck!

basicly the other guy acts irrational... he is punishing you (and himself) and rewarding the folders so acutally de NE still holds because he is acutally giving up an edge...

so i think this is pretty much a case of implicit collusion because he has the ability to punish you...

you can deviate from the NE to gain upper hand again but then it goes into that "he thinks that i think" game...

the point of it is though that he cannot use it to gain more EV! he is giving it up no matter what you do, but its also costing you more than him. he basicly has the power to move your EV to others

i think this is an important lesson that you can put yourself into a position where you cannot win just because another player wants it to be like that

maybe one should calculate the version if you thighten up a bit and the other is max. exploiting you and compare your EV to the one where he is spite-calling you

my intuition says that your EV if adjusting and beeing maximaly exploited is lesser than if you get spite-called

[plexiq]

@Paxinor
Well, the Caller can not only punish the loose pusher. If we assume a non-robotic Pusher, who will adjust to the loose calling, the Callers EQ will in fact significantly increase.

In the example, if BB is calling w/ 25% instead of 8.5%, he is giving up $EV of 0.013, while the Pusher loses almost about 15 times as much.

If the Pusher correctly adjust to this wider call range, he can only Push 23.5%. In this new state, the Callers $EV is 0.25 higher than the original NE state.

@juk: Posted before seeing your reply, but the above kind of answers some of your question. I can share that code, if you guys are interested, although its pretty messy [img]/images/graemlins/wink.gif[/img]

[jukofyork]

[ QUOTE ]
Ok, here is a toy game featuring a "spite-calling" situation:

Basic game is the same as in Math of Poker, pg 127.

*) Every player is dealt a hand in [0...1].
*) SB ("Pusher") can push or fold
*) If SB pushes, BB ("Caller") can call or fold.
*) If there is a showdown, the player with the higher hand has 2/3 pot equity.

We use stacks of 5BB (SB=0.5, BB=1).

So far, thats just "normal" HeadsUp - and there s no possible spite-calling. After all, we are still in a zero-sum game atm. The NE for this "base game" is: Pusher: 70%, Caller: 56%.

Now lets add the possibility to "spite call":
We now change the game, such that the players will convert their stacks to money after the game, and the players goal is to optimize their $EV. However, the conversion is non-linear. Their stack will be converted to money by payout(chips)=sqrt(chips). (Any strictly growing function will do, as long as it grows "slower than linear". Sqrt is an arbitrary choice.)

This models to some degree the situation of an SNG, because doubling up in chips will now be worth less than double $.

In this modified game, the NE would be:
Pusher: Top 100%, Caller: 8.6%.

We are only 5BB deep, and NE suggests that BB is only calling 8.6% against an ATC push. Alright so far [img]/images/graemlins/smile.gif[/img]



In the plot you can see that the Caller can deal huge "EV-damage" to the pusher, by sacrificing very little EV himself. I think that the NE is unsuitable in this situation, because the caller could clearly "force" the pusher into a more favorable state.

[/ QUOTE ]
Great post! If you have the code at hand and it's easy to edit, could you try something:

Iterate through each call range from 0.0 to 1.0 in 0.01 graduations and find the best-response strategy for the pusher along with the EV for the caller vs this best-response strategy. Then plot the EVs for each call range and also find the optimal "spite calling equilibrium" range for the caller.

How does the optimal "spite calling equilibrium" calling/pushing ranges compare to the NE calling/pushing ranges? How do the EV's compare for both players?

Juk [img]/images/graemlins/smile.gif[/img]