proper sample size for data?

[NthingToLose]

I have been keeping real strong records in Pokertracker and SNG Tracker for awhile now and I'm curious, what is a proper sample size to determine something like an accurate ROI from one table SNGs? Or an accurate distribution of finishes to determine leaks in my game? Thanks

[nebben]

It depends on your Standard Deviation. The smaller your SD, the fewer games you need for a sample. I believe common for SNGS is 2k+

[NthingToLose]

2k sample size? Is anyone able to confirm this with empirically trying it? Just curious seems like alot to find an accurate ROI

[pzhon]

[ QUOTE ]
I have been keeping real strong records in Pokertracker and SNG Tracker for awhile now and I'm curious, what is a proper sample size to determine something like an accurate ROI from one table SNGs? Or an accurate distribution of finishes to determine leaks in my game? Thanks

[/ QUOTE ]
People will give you a lot of specific numbers without meaning. To choose a number, you need to say how accurately you want to determine the statistics. A rough 95% confidence interval after n tournaments is the observed result +- 340%/Squareroot(n), since the standard deviation for one tournament is about 170% of a buy-in.

People often say that they can tell something from the place distribution. I'm skeptical; does placing 3rd a lot mean you emphasize surviving the bubble, or ignore the difference between second and third? Anyway, if you want to determine a confidence interval for your ITM percentage, a 95% confidence interval is about the observed rate +- 95%/Sqrt(n), since the standard deviation per tournament is just under 50%.

[NthingToLose]

Pzhon, would you be willing to elaborate more? I'm sorry I have an extremley low knowledge of this subject and am starting to understand but, I'm afraid I am not all the way there yet. Just don't understand how you got the standard deviation in the first part of your response nor am I following the equation. Thanks sorry for the ignorance [img]/images/graemlins/smile.gif[/img]

[LCposter]

Ok, I'll try to do a quick primer of the math involved. I apologize if I'm repeating things you already know.

The standard deviation for an SNG is primarily a function of the payout structure and to a lesser extent your finish distribution. As long as you remember that variance = E(x^2) - E(x)^2, you can calculate your own SD based on your stats.

I'll give an example for party $10+1 (10 players, 50/30/20 payout), finish distribution 15%/13%/12%. First off, realize that even though the payouts are 50/30/20/0, you have to subtract the buyin+rake so that net payouts are 39/19/9/-11.

E(x) = 0.15 * 39 + 0.13 * 19 + 0.12 * 9 + 0.60 * (-11) = 2.8
E(x^2) = 0.15 * 39^2 + 0.13 * 19^2 + 0.12 * 9^2 + 0.60 * (-11)^2 = 357.4
Var(x) = E(x^2) - E(x)^2 = 357.4 - 2.8^2 = 349.56
SD(x) = Sqrt(Var(x)) = Sqrt(349.56) = 18.69652
SD(x)/Buy-in = 18.69652 / 11 = 169.9684%

That's why pzhon quoted an SD around 170%. You can tweak the first, second, and third place finish percentages, as long as you keep the ROI reasonable (i.e. not at the -100% or +354% extremes) it doesn't change the SD that much.

For independent trials, the variances add (i.e. V(x + y) = V(x) + V(y)).
Therefore SD(profit) after n games = 18.69652 * sqrt(n).
And SD(profit)/Buy-in = 18.69652 * sqrt(n) / 11n = 170% / sqrt(n)

To form a 95% confidence interval, we need to go +/- 2 standard deviations from the estimate,
which is why pzhon multiplied by 2 to get 340%/sqrt(n).

Finally, we can do the algebra to solve for n.

To be 95% confident your ROI estimate is within 5% of your "true" ROI:

5% = 340% / sqrt(n)
sqrt(n) = 68
n = 68^2 = 4624

To be 95% confident your ROI estimate is within 1% of your "true" ROI:

1% = 340% / sqrt(n)
sqrt(n) = 340
n = 340^2 = 115,600

I'm not 100% sure about these last calculations. Someone may want to confirm whether they are correct or not.

I know it's discouraging, but the nature of SNGs (and most forms of poker for that matter) is that the variance per trial is quite large, so you need several thousand trials before you can have much confidence about your ROI. At the same time, the conditions would need to remain static during this stretch of several thousand SNGs. The reality is that the game conditions (your skill, opponents' skill, etc.) change too rapidly to ever know what your "true" ROI is. But in any case, it is nice to know the 95% confidence interval for your ROI assuming static game conditions.

[Lexander]

Those calculations look correct. It also shows very well I think why the 5% significance level is often used, since that 1% level has no power except at huge sample sizes.

[NthingToLose]

Thank you! Those were the mathmatical justifications and explanation I needed to understand why... Ya daunting to say the least on the amount of SNGs needed to get that type of SD. Thanks again.