Fictitious play for multi-player games
 

If fictitious play is used to compute a NE in a multi-player game where it is possible for a player to "spite" another (such as in SNGs), then is it correct to assume that each player will attempt to only rationally maximise their own EV for the update rule?

This would mean that you would roughly use this update algorithm:

1. Init strategy to something arbitrary (A).
2. Find the maximal exploitative strategy to A (B).
3. Find the maximal exploitative strategy to a player who plays A or B each with a 50% chance of being selected (C).
4. Find the maximal exploitative strategy to a player who plays A or B or C each with a 33.3% chance of being selected (D).
5. Find the maximal exploitative strategy to a player who plays A or B or C or D each with a 25% chance of being selected (E).
.
.
.
N. Stop when no further exploitative strategy can be found against the strategy collection (or until some reasonable exploitative EV threshold is reached).

This is the basic idea that the ICM Nash Calculator is using.

But, by just assuming that each player will attempt to rationally maximise their own EV seems to ignore the fact that they could also gain EV by another method: You could give up some of your own EV to cost an opponent even more which in turn would force the opponent to change their strategy possibly meaning that you gain EV by threatening to give up some (ie: by partially minimizing your opponent's EV).

This seems to fly in the face of what I know about NE states though, as it shouldn't be possible for a player to deviate profitably from a NE (assuming the above algorithm really does converge to a NE).

Is the reasoning flawed here somewhere? If so, can you think of a simple game as an example?

Is the above algorithm flawed? If so, then what alteration would be required for the update rule?


This post is related to a post in the STT forum which discusses taking a -EV play now to force your opponent to alter his strategy so as to possibly gain more EV in the future. The reason this might be correct is that the play only costs you ~$30, yet costs your opponent ~$300:

http://forumserver.twoplustwo.com/sh...age=0&vc=1

This led me to wondering what is the correct method to compute a NE where it's possible to "spite" an opponent like this:

http://forumserver.twoplustwo.com/sh...age=0&vc=1

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

[ QUOTE ]
This seems to fly in the face of what I know about NE states though, as it shouldn't be possible for a player to deviate profitably from a NE (assuming the above algorithm really does converge to a NE).

[/ QUOTE ]

Well, the definition of a NE is still satisfied. No player can unilaterally deviate from the NE to gain value. Thats true in your example - if only 1 player deviates, he cant improve.

If you assume bot-like players who wont deviate from the NE no matter what happens, then your best choice is to play the NE as well.

That said, i think the problem you describe is inherent in the definition of the NE. It doesnt really matter what algorithm we would use to find (or approximate) the NE.

Some kind of "raw" first idea, didnt really think it through yet:
Instead of optimizing the "current" equity (ie, playing maximally exploitative), each player tries to "drag" the strategies in a direction that will give him better equity than the current state - but only as long as his deviation from maximally exploitative play costs the respective opponents more EV than him.

This should converge to a more "robust" set of strategies. But then, these strategies will be easily exploitable by opponents who simply skip their "spite calls".

This gets pretty interesting if you think about it. If we draw random players from a population of 50% NE, and 50% "spite callers" and put them into a game, the spite caller population would have a higher expectation in this game, i think.

Need to think it through before posting any more. I hope the above makes any sense, lol.

Re: Fictitious play for multi-player games
 

Since poker is a mixed strategy game. There will be multiple NE. One set of decisions is not the right play but mixing up your strategy is more profitable because it is a repeated game.

Anything that makes your opponent play differently, that you can take advantage of in the future is optimal and if you are able to do it you should. However, that's harder to do online then live.

Re: Fictitious play for multi-player games
 

I think the difference lies in the definition of the "game." Fictitious play will work (I've used it) if you assume the current hand is a one-shot deal. There are no more interactions after the current hand. However if you extend the definition of the game to cover multiple hands then it gets a lot more complicated.

A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

Tysen

Re: Fictitious play for multi-player games
 

[ QUOTE ]
Some kind of "raw" first idea, didnt really think it through yet:
Instead of optimizing the "current" equity (ie, playing maximally exploitative), each player tries to "drag" the strategies in a direction that will give him better equity than the current state - but only as long as his deviation from maximally exploitative play costs the respective opponents more EV than him.

[/ QUOTE ]
Yep, this is what I was thinking, but "dragging" the values could be very computationally expensive to try. The basic idea would be to somehow "drag" your own strategy into the space where it is -EV for you and see how that effects your opponents maximally exploitative strategy. The current update rule never considers these -EV calls.

Perhaps rather than "dragging" this could be accomplished by some kind of recursive update rule which is about order O(n) more complex? One idea would be to find the gradient of EV change for you for each variable of the strategy and then update your strategy variable by moving in the direction which increases EV for you (as opposed to updating it based on whether it is +EV or -EV for you to play against the current opposing strategy).

I've still not thought about this much yet so the idea might be flawed or there might be a much simpler way to combine the maximally exploitative strategy with the maximally spiteful strategy and update the rules based on both.

[ QUOTE ]
This should converge to a more "robust" set of strategies. But then, these strategies will be easily exploitable by opponents who simply skip their "spite calls".

[/ QUOTE ]
I don't think it could really be exploited, as it's the threat of the spite calls more than the calls itself that's important. The equilibrium should mean that if player A deviates by not spite calling player B anymore then player B won't be making the pushes that are punished by the spite calls anyway so nothing has changed. If the player B decides to push these anyway knowing that he'll be spite called then he's just made his strategy -EV compared to if he respected the player A's spite calls.

[ QUOTE ]
This gets pretty interesting if you think about it. If we draw random players from a population of 50% NE, and 50% "spite callers" and put them into a game, the spite caller population would have a higher expectation in this game, i think.

[/ QUOTE ]
That's quite interesting and would make an interesting experiment. What would happen if you tried to train up a maximally exploitative strategy to play against this mixed NE/spite player? Perhaps this would be a more robust strategy than NE alone?

[ QUOTE ]
Need to think it through before posting any more. I hope the above makes any sense, lol.

[/ QUOTE ]
Yep, some of my ideas might be totally off here too - I've just woke up and not really thought too carefully about all this yet, but overall it makes for some interesting thinking!

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

[ QUOTE ]
I think the difference lies in the definition of the "game." Fictitious play will work (I've used it) if you assume the current hand is a one-shot deal. There are no more interactions after the current hand. However if you extend the definition of the game to cover multiple hands then it gets a lot more complicated.

A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

[/ QUOTE ]
Yep, I guess this would require expanding the game tree out to be able to see the blinds moving and the big stack getting into more and more +EV bullying situations. Perhaps it could be expanded into the next hand (or even next few hands) and still be computationally tractable? Not sure how much better the solutions would be though.

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

Kind of forgot about this thread, sorry [img]/images/graemlins/smile.gif[/img]

[ QUOTE ]
A perfect example is when there is a big stack bullying the table near the bubble. Nash says the big stack should raise almost every hand and the small stacks should almost always fold. However just being in this situation is -EV for the small stacks. It would be in a small stacks long-term interest to call more liberally and punish the raiser. This will attempt to get the big stack to stop his bullying. But you have to take a -EV move now in order to try and stop being in continually -EV situations in the future.

But that would be way too complicated to figure out with a computer...

Tysen

[/ QUOTE ]

I think the example is actually mixing in a different problem.

One part of the problem you describe boils down to flaws of ICM. ICM overestimates midstack-equities at the bubble, and underestimates bigstack equity. If we had access to a better EQ-estimation, midstacks would automatically call wider, because relative equities of folding/busting/doubling up would change.

With ICM we have lots of scenarios where players are expected to win/lose equity during the next orbit. This should never be the case with an accurate EQ model.

As i understand it, thats to be seen "separated" from our original problem: That the NE is usually a bad state for the caller, because he is actually in the position to "force" the pusher into a more favorable state. I think this is a problem with the NE altogether. Maybe i can think of some toy game to better demonstrate my though,...(hopefully i wont forget about the thread, again [img]/images/graemlins/laugh.gif[/img])

Re: Fictitious play for multi-player games
 

Ok, here is a toy game featuring a "spite-calling" situation:

Basic game is the same as in Math of Poker, pg 127.

*) Every player is dealt a hand in [0...1].
*) SB ("Pusher") can push or fold
*) If SB pushes, BB ("Caller") can call or fold.
*) If there is a showdown, the player with the higher hand has 2/3 pot equity.

We use stacks of 5BB (SB=0.5, BB=1).

So far, thats just "normal" HeadsUp - and there s no possible spite-calling. After all, we are still in a zero-sum game atm. The NE for this "base game" is: Pusher: 70%, Caller: 56%.

Now lets add the possibility to "spite call":
We now change the game, such that the players will convert their stacks to money after the game, and the players goal is to optimize their $EV. However, the conversion is non-linear. Their stack will be converted to money by payout(chips)=sqrt(chips). (Any strictly growing function will do, as long as it grows "slower than linear". Sqrt is an arbitrary choice.)

This models to some degree the situation of an SNG, because doubling up in chips will now be worth less than double $.

In this modified game, the NE would be:
Pusher: Top 100%, Caller: 8.6%.

We are only 5BB deep, and NE suggests that BB is only calling 8.6% against an ATC push. Alright so far [img]/images/graemlins/smile.gif[/img]

http://www.holdemresources.net/misc/spitecalling.png

In the plot you can see that the Caller can deal huge "EV-damage" to the pusher, by sacrificing very little EV himself. I think that the NE is unsuitable in this situation, because the caller could clearly "force" the pusher into a more favorable state.

Re: Fictitious play for multi-player games
 

to simulate a sit n go properly, wouldn't it be suitable to create a game where the sum of $EV is always the same? i mean this is the crucial point, because it needs to be a zero sum game! and ICM is a zero sum game too...

Re: Fictitious play for multi-player games
 

What we want to simulate here, is the SB-vs-BB "subgame", after n other players folded. ICM isnt zero sum in this situation (if we only look at the involved players).

Re: Fictitious play for multi-player games
 

ah ok know i get it...

well this is why multiplayer games suck!

basicly the other guy acts irrational... he is punishing you (and himself) and rewarding the folders so acutally de NE still holds because he is acutally giving up an edge...

so i think this is pretty much a case of implicit collusion because he has the ability to punish you...

you can deviate from the NE to gain upper hand again but then it goes into that "he thinks that i think" game...

the point of it is though that he cannot use it to gain more EV! he is giving it up no matter what you do, but its also costing you more than him. he basicly has the power to move your EV to others

i think this is an important lesson that you can put yourself into a position where you cannot win just because another player wants it to be like that

maybe one should calculate the version if you thighten up a bit and the other is max. exploiting you and compare your EV to the one where he is spite-calling you

my intuition says that your EV if adjusting and beeing maximaly exploited is lesser than if you get spite-called

Re: Fictitious play for multi-player games
 

[ QUOTE ]
Ok, here is a toy game featuring a "spite-calling" situation:

Basic game is the same as in Math of Poker, pg 127.

*) Every player is dealt a hand in [0...1].
*) SB ("Pusher") can push or fold
*) If SB pushes, BB ("Caller") can call or fold.
*) If there is a showdown, the player with the higher hand has 2/3 pot equity.

We use stacks of 5BB (SB=0.5, BB=1).

So far, thats just "normal" HeadsUp - and there s no possible spite-calling. After all, we are still in a zero-sum game atm. The NE for this "base game" is: Pusher: 70%, Caller: 56%.

Now lets add the possibility to "spite call":
We now change the game, such that the players will convert their stacks to money after the game, and the players goal is to optimize their $EV. However, the conversion is non-linear. Their stack will be converted to money by payout(chips)=sqrt(chips). (Any strictly growing function will do, as long as it grows "slower than linear". Sqrt is an arbitrary choice.)

This models to some degree the situation of an SNG, because doubling up in chips will now be worth less than double $.

In this modified game, the NE would be:
Pusher: Top 100%, Caller: 8.6%.

We are only 5BB deep, and NE suggests that BB is only calling 8.6% against an ATC push. Alright so far [img]/images/graemlins/smile.gif[/img]

http://www.holdemresources.net/misc/spitecalling.png

In the plot you can see that the Caller can deal huge "EV-damage" to the pusher, by sacrificing very little EV himself. I think that the NE is unsuitable in this situation, because the caller could clearly "force" the pusher into a more favorable state.

[/ QUOTE ]
Great post! If you have the code at hand and it's easy to edit, could you try something:

Iterate through each call range from 0.0 to 1.0 in 0.01 graduations and find the best-response strategy for the pusher along with the EV for the caller vs this best-response strategy. Then plot the EVs for each call range and also find the optimal "spite calling equilibrium" range for the caller.

How does the optimal "spite calling equilibrium" calling/pushing ranges compare to the NE calling/pushing ranges? How do the EV's compare for both players?

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

@Paxinor
Well, the Caller can not only punish the loose pusher. If we assume a non-robotic Pusher, who will adjust to the loose calling, the Callers EQ will in fact significantly increase.

In the example, if BB is calling w/ 25% instead of 8.5%, he is giving up $EV of 0.013, while the Pusher loses almost about 15 times as much.

If the Pusher correctly adjust to this wider call range, he can only Push 23.5%. In this new state, the Callers $EV is 0.25 higher than the original NE state.

@juk: Posted before seeing your reply, but the above kind of answers some of your question. I can share that code, if you guys are interested, although its pretty messy [img]/images/graemlins/wink.gif[/img]

Re: Fictitious play for multi-player games
 

Here s the code. Possibly quite buggy, was coding this while trying to construct a toy game. (You need R to run this: www.r-project.org/ )

<font class="small">Code:</font><hr /><pre>
#our "equity transformation"
chipTrans&lt;-function(chips=0){return(chips^0.5)}

#stack sizes
stacks = 5

#winning percentage of range1 vs range2
winPct&lt;-function(r1=1,r2=1)
{
return((0.5*min(r1,r2)+0.66*max(r2-r1,0)+0.34*max(r1-r2,0))/max(r1,r2))
}

#EV of pusher, p..pushrange, c..callrange
EVPusher &lt;- function(p=1,c=1)
{
return (
(1-p)*chipTrans(stacks-0.5)+
p*(
c*(
winPct(p,c)*chipTrans(2*stacks) +
(1-winPct(p,c))*chipTrans(0)
)+
(1-c)*chipTrans(stacks+1)
)
)
}

#EV of caller, p..pushrange, c..callrange
EVCaller &lt;- function(p=1,c=1)
{
return (
(1-p)*chipTrans(stacks+0.5)+
p*(
c*(
winPct(c,p)*chipTrans(2*stacks) +
(1-winPct(c,p))*chipTrans(0)
)+
(1-c)*chipTrans(stacks-1)
)
)
}

#One iteration of fictitious play
nextIter&lt;-function(s=c(1,1), weight=1)
{
bestcev=0
bestpev=0

bestp=-1
bestc=-1

pos = (1:1000)/1000
for(i in c(1:length(pos)))
{
thisc = EVCaller(s[1],pos[i])
thisp = EVPusher(pos[i],s[2])

if(thisc&gt;bestcev){
bestcev=thisc
bestc=pos[i]
}

if(thisp&gt;bestpev){
bestpev=thisp
bestp=pos[i]
}

}


sn = c(bestp,bestc)
return(weight*sn+(1-weight)*s)
}

#300 iters of fictitious play
runFictitiousPlay&lt;-function()
{
# r[0]: push range, r[1]: call range
r=c(1,0)
for(w in 1/(1:300))
{
r=nextIter(r,w)
print(r)
}
return(r)
}


#cr = (1:1000)/1000

###plotting EQ vs variable callrange
#pev = apply(X=t(cr),FUN=EVPusher,p=r[1],MARGIN=2)
#cev = apply(X=t(cr),FUN=EVCaller,p=r[1],MARGIN=2)
#plot(cr,cev,ty='l', lty=2,ylim=c(2.2,3.5))
#lines(cr,pev)

###plotting EQ vs variable pushrange
#pev = apply(X=t(cr),FUN=EVPusher,c=r[2],MARGIN=2)
#cev = apply(X=t(cr),FUN=EVCaller,c=r[2],MARGIN=2)
#plot(cr,cev,ty='l', lty=2)
#lines(cr,pev)
</pre><hr />

Re: Fictitious play for multi-player games
 

[ QUOTE ]
If the Pusher correctly adjust to this wider call range, he can only Push 23.5%. In this new state, the Callers $EV is 0.25 higher than the original NE state.

[/ QUOTE ]
Ah, but this is deceptive. Most of the Caller's EV gain is the fact that he gets a walk 76.5% if the time. But those times where the Pusher does push, it's a different situation. Once the push is made, it's not optimal for the caller to call so often. In fact he should be calling LESS often than if the Pusher goes 100%. It's like the prisoner's dilemma. Both would be better off by cooperating, but no matter how your opponent plays, you're better off with defecting.

Tysen

Re: Fictitious play for multi-player games
 

In case I take up playing SNGs again, I may use a +EV screen name like, "SpiteCa11er." Then I probably won't do it.

I would use a weighted average, not an equal average, but I don't think it makes much of a difference since the algoritm is so fast and can compute hundreds of iterations.

Given a 50-50 population of spite callers and a population of NE players, it is not clear that the spite callers would win on average. The spite callers will make spite calls against each other, too.

Re: Fictitious play for multi-player games
 

@trojanrabbit
After the push is made, there is no value in calling looser (if we assume that looser calling will not effect the pushers future range) - right.

I understand the problem you describe, and its perfectly correct of course. But i still think the NE is not a good solution for "practical use" here, as its so vulnerable to deviations in the callers strategy, while varying the calling strategy a bit is basically EV-neutral.

Considering Spite-Caller vs NE population:
I think Spite callers would only perform better if they also adjust their push-range to be tighter. They give up some EV against the NE players, but perform vastly better against other spite callers. But i guess that has to be tested, maybe im missing something.

[ QUOTE ]
I would use a weighted average, not an equal average, but I don't think it makes much of a difference since the algoritm is so fast and can compute hundreds of iterations.

[/ QUOTE ]

Yeh, im using a weighted average for the Nash Calculator page. But in this case i preferred to keep things as straightforward as possible [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

@ plexiq: true if the pusher would adjust to his callrange then his EV would be higher but:

does the pusher acutally wants do adjust? in gametheory usually you assume that strategies are common knowledge and therefore are always exploited which is clearly not true to poker

but assuming that as soon as you thighten up your range basicly he is gonna tighten up too in his calling range. so you give him a walk more often and he also adjust to your calling range

my question is: might this be a better option for the pusher? if so he has motivation to deviate

else he is gonna be punished but basicly can do nothing about it

(this is strict game theory thinking)

now if you go to practice where strategies are acutally hidden it might acutally be good to deviate because the opposite player cannot adjust prefectly to your new strategy...

but i just wonder from a strict theoretic standpoint and i really doubt that he should deviate if the caller knows his strategy...

Re: Fictitious play for multi-player games
 

I guess the challenge here is:
Can we model the "strategy-negotiation process" in a way, such that the Caller eventually "realizes" that its profitable for him to stay at a wider call range, given that the pusher is adopting his ranges?

Even when the pusher is pushing tighter now and switching to a tighter call range would be immediately profitably for the caller - the idea is to simply to "keep" the pusher from loosening up again, because tightening up will eventually end up in a worse state for the caller.

Re: Fictitious play for multi-player games
 

[ QUOTE ]
I guess the challenge here is:
Can we model the "strategy-negotiation process" in a way, such that the Caller eventually "realizes" that its profitable for him to stay at a wider call range, given that the pusher is adopting his ranges?

Even when the pusher is pushing tighter now and switching to a tighter call range would be immediately profitably for the caller - the idea is to simply to "keep" the pusher from loosening up again, because tightening up will eventually end up in a worse state for the caller.

[/ QUOTE ]
Ah, I'd forgot that the wider calling, leading to winder pushing, would end up with the caller being able to exploit by calling thinner the the original NE (I was thinking that it would just be like NE+SpiteCall for some reason). So sadly this means that their will be no "spite-call" equilibrium possible.

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

[ QUOTE ]
[ QUOTE ]
I guess the challenge here is:
Can we model the "strategy-negotiation process" in a way, such that the Caller eventually "realizes" that its profitable for him to stay at a wider call range, given that the pusher is adopting his ranges?

Even when the pusher is pushing tighter now and switching to a tighter call range would be immediately profitably for the caller - the idea is to simply to "keep" the pusher from loosening up again, because tightening up will eventually end up in a worse state for the caller.

[/ QUOTE ]
Ah, I'd forgot that the wider calling, leading to winder pushing, would end up with the caller being able to exploit by calling thinner the the original NE (I was thinking that it would just be like NE+SpiteCall for some reason). So sadly this means that their will be no "spite-call" equilibrium possible.

[/ QUOTE ]
OK, I've thought about this some more and how about if we add the extra constraint that the caller is forced to state his exact strategy before the pusher pushes?

This should get round the possibility of the caller to pass up on his spite calls and allow for convergence to a "spite-call" equilibrium?

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

there is definitly no "spite-call" equilibrium.

it doesn't help to fix the callers strategy. then you just use the maximaly exploiting counterstrategy as a pusher. but then its not even a equilibrium since there is only one strategy...

so bascily the other guy is just moving out of the NE trying to make you adjust to an exploiting strategy and go into a kind of "tit for tat" game.

but its really exciting to see that players have GREAT power over the game and it's outcome if they do not choose to max payout.

however strictly theoretic, there is no "spite-call" equilibrium whatsoever. even if you add constraints...

if someone moves away from NE and the other adjusts somehow, there will be no equilibrium. so you cannot just lay back, play your strategy and wait that the opponent makes a mistake. then your in the "he thinks that i thinks that he thinks" not that the state cannot be profitably played

in pracice i think he makes a good move... i mean you should adjust. you improve your EV by that and he improves his. because your strategy will not be public he cannot exploit you perfectly. therefore it is very likely that you will turn out with a better EV than in the NE but he will too!

so actually we have the kind of paradox situation that because he cannot exploit you perfectly because he doesn't know your exact strategy he can improve his EV!! this is pretty sick if you think about it because this goes so much agaisnt intuition.

well (while im writing it) acutally i think there is some kind of equilibrium:

take all possible strategies of the caller and let them be exploitet perfectly by the pusher. there is an EV for all of those situations. if some of those EVs are for the caller are above his NE EV then he should just choose that strategy and risk to get maximaly exploitet.

of course the pusher then as a 23% pushing range and could be exploitet again by thigten up. but the you get into that tit for tat game where when it starts to ge sophisticated in strategy changing, NE strategy would be best again

so you simply don't adjust! you giving up the edge of counterexploiting him to preserve the edge you gain by getting maximaly exploitet over the NE EV (this sounds pretty sick)

so if the calculations of plexiq are correct, the other guy starts calling all the time and is therefore transfering EV to the folders. if you are a thinking player you adjust.

you just push 23% of your hands. good for you, you improved your EV again. but the loose caller improved his too! (if the calculations are correct). so he should keep his strategy up. and there you have a kind of equilibrium where strategys will not be changed due to lack of strategys that are known and the problem that you might loose in a tit for tat game.

well this is really really interesting!

Re: Fictitious play for multi-player games
 

[ QUOTE ]
OK, I've thought about this some more and how about if we add the extra constraint that the caller is forced to state his exact strategy before the pusher pushes?

[/ QUOTE ]

This doesn't work either. If I'm Caller, I'll say "I'm calling 100% and you know I'm stuck with it." Now Pusher has to fold almost everything since it's just the same problem in reverse. It's the right of first bluff.

Tysen

Re: Fictitious play for multi-player games
 

[ QUOTE ]
[ QUOTE ]
OK, I've thought about this some more and how about if we add the extra constraint that the caller is forced to state his exact strategy before the pusher pushes?

[/ QUOTE ]

This doesn't work either. If I'm Caller, I'll say "I'm calling 100% and you know I'm stuck with it." Now Pusher has to fold almost everything since it's just the same problem in reverse. It's the right of first bluff.

[/ QUOTE ]
It does mean that the strategies will converge to something stable (not necessarily sensible though) and as you put it "right of first bluff" here is just the caller stating his spite-calling intentions before the hands starts. The really interesting thing I'd love to know is the answer too is this:

If we try all possible calling ranges (where the caller has to state his calling range first) and the pusher then computes his maximally exploitative strategy against each of these calling ranges, then the caller settles on the calling range which is most +EV for him. Will this strategy be more +EV for the caller than the strategy found by each attempting to maximally exploit each other's play (ie: the Nash Equilibrium found by fictitious play)?

If it does turn out so that the "spite-calling equilibrium" is more +EV for the caller and we assume that any attempt to pass on the spite calls will be seen instantly by the pusher and adapted for (hence the extra constraint of stating the caller's strategy before the hands starts to allow for convergence), then it would seem that the "spite-calling equilibrium" is the better strategy for the caller to follow and deviating from it towards the NE would be a bad idea.

If nobody answers this before xmas, then most likely I'll spend a few days getting the code together to try and answer this for SB vs BB situations.

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

@juk:
The problem with your approach is imo, that pusher &amp; caller are treated differently. The caller is basically playing some non-zero-sum version of minimax, while the pusher sticks to maximally exploiting the callers strategy. I guess we would want to use the same assumptions for both players?

But i ll run the numbers on this later, definitely before xmas [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

[ QUOTE ]
@juk:
The problem with your approach is imo, that pusher &amp; caller are treated differently. The caller is basically playing some non-zero-sum version of minimax, while the pusher sticks to maximally exploiting the callers strategy. I guess we would want to use the same assumptions for both players?

[/ QUOTE ]
Yep, I suppose it might be just as valid to consider "spite-pushing". Whether it's really possible in real SNGs, I don't know, but I guess another toy game could be made where the pusher could also spite.

[ QUOTE ]
But i ll run the numbers on this later, definitely before xmas [img]/images/graemlins/smile.gif[/img]

[/ QUOTE ]
Cool! [img]/images/graemlins/smile.gif[/img]

Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

Depending on how you set up the game, whomever can "lock in" their action first will push 100%. That lock can come in the form of pushing your chips into the middle of the table or some artificially binding strategy declaration. There's nothing that the other player can do about this and it is the only stable equilibrium you can find. I firmly believe that the only way you'll be able to solve for spite calling is if you consider the EV of future hands beyond this one. That's what I'm going to be looking into.

Tysen

Re: Fictitious play for multi-player games
 

[ QUOTE ]
Depending on how you set up the game, whomever can "lock in" their action first will push 100%. That lock can come in the form of pushing your chips into the middle of the table or some artificially binding strategy declaration. There's nothing that the other player can do about this and it is the only stable equilibrium you can find. I firmly believe that the only way you'll be able to solve for spite calling is if you consider the EV of future hands beyond this one. That's what I'm going to be looking into.

[/ QUOTE ]
A bit OT, but I managed to get a copy of your new book and from initial impressions it looks awesome! It's basically what I had hoped both Chen's "Mathematics of Poker" and Moshmans's "Sit 'n Go Strategy" would be about: applicable theory, as opposed to toy-game theory and/or endless HOH-style hand examples.

Great work! Juk [img]/images/graemlins/smile.gif[/img]

Re: Fictitious play for multi-player games
 

[ QUOTE ]
A bit OT, but I managed to get a copy of your new book and from initial impressions it looks awesome! It's basically what I had hoped both Chen's "Mathematics of Poker" and Moshmans's "Sit 'n Go Strategy" would be about: applicable theory, as opposed to toy-game theory and/or endless HOH-style hand examples.

Great work! Juk [img]/images/graemlins/smile.gif[/img]

[/ QUOTE ]

Thanks, Juk! I've heard that before that KE is the book people were hoping to get when they bought those two. Perhaps some of this effort here can make its way into another volume... [img]/images/graemlins/tongue.gif[/img]

Tysen