How many games for a reliable ROI?

[Goldmund]

How many games is a good number to rely upon to conclude you are A) a winning player and B) that your ROI is more or less reliable. Do 200 games give a good indication of winrate at a certain level? One hundred? Goldmund

[jay_shark]

Suppose you've played 100 games and you've won 60 of them . The question you want to know is what's the probability that i'm within lets say 5 % of my true win rate .

Here I will use the binomial probability formula .

P = nCk *p^k(1-p)^q

k = success
q= failure
k+q=n

Now we have to plug in values for k=55, 56 , 57 ...65 since you'd like to know within 5 %.

Plug these values for k and you get the following

100c55*0.6^55*0.4^45 =0.047
100c56*0.6^56*0.4^44 = 0.057
100c57*0.6^57*0.4^43= 0.066
100c58*0.6^58*0.4^42= 0.074
100c59*0.6^59*0.4^41=0.079
100c60*0.6^60*0.4^40= 0.08
100c61*0.6^61*0.4^39= 0.0798
100c62*0.6^62*0.4^38= 0.075
100c63*0.6^63*0.4^37= 0.068
100c64*0.6^64*0.4^36= 0.059
100c65*0.6^65*0.4^35=0.049

Now we simply add up all the numbers from the right hand side to get the probability that you're within 5 % of your true win rate after 100 trials .

The probability becomes 73.3% that you're within 5 % of your true win rate .

If you play 200 games then this number shoots straight up . If you're satisified with having 73% confidence that you're within 5 % of your true win rate (which you shouldn't be), then 100 games is suffucient . Personally it's better you play several hundred games to get a better approximation .

[Goldmund]

Thanks Jay...so what IS the probability after 200 games? Just call me lazy and hey, I went to law school. Goldmund

[pocketjacks]

As soon as you asked the question the number 200 came to my mind.

[jay_shark]

Unfortunately my calculator sucks and it doesn't go that high .

If anyone else has a better calulator , just plug these numbers for k=110 ,111, ... 119 , 120 .

You can also do better than 5 % . You may want to ask the probability you're within 3 % of your true win rate . In this case you plug in values for k=114...126

do you see why you plug these values for k ?

[Goldmund]

Not yet Jay, I'm at work right now, but I''ll delve into it tonight...But why is it that for HU sng's a much smaller sample is judged to be sufficient proof of winrate compared to cashplay?

One hundred games only constitute like 6000-8000 hands, depending how long your game lasts on average. That number of hands is considered HIGHLY inadequate to say anything meaningful about your cashgame winrate, even 20K hands, so like 300 HU sng's is, a small sample.

Obviously variance is lower, but is it really THAT much lower? Goldmund


[ QUOTE ]
Unfortunately my calculator sucks and it doesn't go that high .

If anyone else has a better calulator , just plug these numbers for k=110 ,111, ... 119 , 120 .

You can also do better than 5 % . You may want to ask the probability you're within 3 % of your true win rate . In this case you plug in values for k=114...126

do you see why you plug these values for k ?

[/ QUOTE ]

[jay_shark]

This is why even when you're playing heads up , you can't be certain that your roi is accurate after 100 hands . If you play 500-1000 hands then you will start to show meaningful results . It is still possible that after 1000 games played , you can have a positive win/loss ratio but still be a losing player . Of course , as you should expect , the probability decreases and the limit approaches zero as n goes to infinity .

[roundhouse]

Sorry to revive a dead thread but this is the sort of question which gets asked a lot, I wanted to produce the table below but only just got around to it.




The confidence level is determined by two things only: your win % and the number of SnGs played. Of the two the latter - number played - makes by far the biggest difference (as you can see).

How to interpret the table? Well lets say you've played 250 HU SnGs at a given level and you've won 55% of them. Looking up the above table you should see this gives you the value 6.2%, this means that you can be 95% sure that your true win rate is in the range 55% +/- 6.2% (i.e. 48.8 to 61.2%). If you are playing with a 50+2.5 type structure, as stars has, you are trying to gain confidence that you are winning more than 52.5% (the break even point); this range dips below 52.5%, so you can't be sure you're a winning player.

Hope that makes it clear.

Cheers,
RH

[dboy23]

nice table. I'm at 221 SNG's of the 10's and 20's combined. At the 10's I was 60% and the 20's I am 65%. I've played about the same number of games at both. When I average them I get 62.44%.

So this means my actual winrate is 62.44 +/- 6.1, giving me a range of 56.34 - 68.54.

That is great news. It means I am very +EV at these things. I still know I can improve though. I will shoot for getting myself into the higher half of that confidence interval.

[Goldmund]

Great work Roundhouse, thanks!