"How many hands do I need to play..."

[Homer]

Introduction

Recently, I've noticed a lot of players asking how many hands they need to play in order to prove that they are a winning player or that they are winning at a certain rate. Usually, the responses received are along to lines of:

- The vague answer -- "You need way more hands than that"

- The random number answer -- "You need at least 100,000,000 hands"

Anyway, I've decided to go over the process of computing confidence intervals, which will allow you to make mathematically valid statements about your true win rate.

Details

In order to compute a confidence interval, you need to locate certain statistics in Poker Tracker, specifically total hands, total table-hours, EV/table-hr (Amount won / Hours played) and SD/hr. All of these statistics are located under Session Notes -> Session Summary, with the exception of standard deviation, which is displayed after a user clicks the "More Detail..." button.

I just started using Poker Tracker again to track my 5/10 SH play, so I'll use those numbers as an example since I was planning to compute the confidence interval for that data anyway.

Hands (h) = 3115
Table-hrs (n) = 32.45
EV/hr (ev) = $39.41
SD/hr (sd) = $182.97

Once you have these statistics, you can use the below formulas to compute the upper bound (u) and lower bound (l) for a given confidence interval (c).

u = ev - normsinv[(1-c)/2]*sd*[1/sqrt(n)]

l = ev + normsinv[(1-c)/2]*sd*[1/sqrt(n)]

For the sample data I set the confidence interval at 95% and with these formulas found that the upper bound is $102 and the lower bound is -$24.

With this information, I can make the following statement:

"I am 95% confident that my true win rate is between -$24 and $102 per table-hr."

-- If I change the confidence interval to...

75%; Lower bound = $2 Upper bound = $76

50%; Lower bound = $18, Upper bound = $61

-- If I keep the confidence interval at 95% and change the hours played to...

100 (9600 hands); Lower bound = $4, Upper Bound = $75

250 (24,000 hands); Lower bound = $17, Upper bound = $62

1000 (96,000 hands); Lower bound = $28, Upper bound = $51

5000 (480,000 hands); Lower bound = $34, Upper bound = $44

10,000 (960,000 hands) Lower bound = $36, Upper bound = $43

From this, you can see that after 3000 hands you can't prove much of anything. It is within the realm of possibility that I have a true win rate of -2.5 BB/hr despite the fact that over 3000 hands I won at a clip of 4 BB/hr. You can also see that even after 500,000 hands, it is not a rarity for a player's true win rate to be 0.5 BB/hr more or less than his results to date.

Creating an Excel spreadsheet

It is quite easy to setup an Excel spreadsheet to compute confidence intervals. Information need only be entered in a few cells, as follows:

Cell A1 - "Hours Played"; Cell B1 - Hours played value (from PT)

Cell A2 - "EV/hr"; Cell B2 - EV/hr value (from PT)

Cell A3 - "SD/hr"; Cell B3 - SD/hr value (from PT)

Cell A4 - "Confidence Interval"; Cell B4 - ".95"

Cell A5 - "Upper Bound"; Cell B5 - <font color="green">=A2-normsinv((1-A4)/2)*A3*(1/sqrt(A1))</font>

Cell A6 - "Lower Bound"; Cell B6 - <font color="green">=A2+normsinv((1-A4)/2)*A3*(1/sqrt(A1))</font>

After you do this, you can mess around with cells A1, A2 and A4 (probably not A3 as your SD/hr converges fairly quickly and isn't likely to change unless your playing style does).

-- Homer

[itsmarty]

Excellent work, thanks for posting it!

Martin

[MicroBob]

Nice post Homey!

Reminiscent of the post you did on streaks (with all the nifty charts) back in the day.

I look forward to constructing my own groovy spreadsheet.

[Homer]

[ QUOTE ]
I look forward to constructing my own groovy spreadsheet.

[/ QUOTE ]

Cool. I just realized I posted this in the Zoo out of habit. Probably would have been better off in Small Stakes or Poker Theory, though I guess it is Internet Gambling related (since I'm talking about using PT stats).

-- Homer

[Lazy Meatball]

Bravo

[SackUp]

Great post for the zoo too!

It could be good for any of the limit stakes as well.

Well done Homer. I'll have to get on this once I get some more hands in /images/graemlins/smile.gif

[MicroBob]

seems to me there are about as many posters asking 'how confident can i be in this win-rate after 2,659 hands??' in the Zoo as in any other forum.

[spamuell]

Thanks Homer, excellent post.

In the forumulae you gave in green, I had to change the As to Bs to get it to work but I think that's probably because I just put the values in different places to where you said.

If I was to change the EV/hr and SD/hr to BB/100 and SD/100 (in BBs), do you know if that would also work?

Thanks again.

[ctv1116]

Excellent analysis as usual.

[Homer]

[ QUOTE ]
In the forumulae you gave in green, I had to change the As to Bs to get it to work but I think that's probably because I just put the values in different places to where you said.

[/ QUOTE ]

No, you did it correctly. I made a mistake. To anyone else who can't get the spreadsheet to work, the upper and lower bound formulas should refer to cells B1, B2 and B4 rather than A1, A2 and A4.

[ QUOTE ]
If I was to change the EV/hr and SD/hr to BB/100 and SD/100 (in BBs), do you know if that would also work?

[/ QUOTE ]

Yes, that will work, provided that instead of using total hours you use sets of 100 hands. In other words, if you've played 10,000 hands at 50 hands/hr, rather than using 200 hrs you would use 100 "sets of 100 hands".

-- Homer

[itsmarty]

That tripped me up as well. Column A is the label and B is the value, so you have to use B's.

I also found out that if you forget the decimal in your confidence interval (95 instead of .95), it really messes things up /images/graemlins/smile.gif

Martin

[itsmarty]

For those without Excel who are interested in trying this out, try OpenOffice (www.openoffice.org). That's what I use for my bankroll spreadsheet, and it handled these additions nicely.

It's free, and mostly compatible with MS Office (only a power user is likely to encounter incompatibilities).

Martin

[RedWave]

I'm pretty much Excel illiterate as far as calculations go, so how do I get the values to appear after I enter all of the information? I put in the values for everything but the equations I enter into cells B5 and B6 don't change to the numbers.

Nevermind, it seems to be working now /images/graemlins/confused.gif

Another stupid question, is EV/hr the same as the win rate/hr?
Nevermind again, the answer is yes and yes the question is stupid. Got it to work, thanks for the great post Homer.

[Jim C]

There is a problem with the given formula. In the original post, you put the numbers in column B, but the formula references the cells in column A. Put the numbers in column A and the labels in column B and the formulae will work.

FWIW,
Jim

[Number4]

Thanks a lot Homer - great post.

[Spicymoose]

What are the limitations on this formula? I have my numbers in, and things seem to look alright, but my .50/1 limit results seem way off.

Now I just started over at a new site (for a rakeback), and only have 674 hands. I have won 17.54 BB/100 hands, with EV/hr of 10.28 and SD/hr of 11.

Now according to your post I can now say that "I am %90 sure that that I will be winning $5-$15.6 at .5/1" which seems completely absurd. My only thought here is that we need the true SD/hr instead of the statistic, which we can never truly have. How big does our sample size need to get before we can use this formula?

Thanks,
Spicymoose

[Alobar]

will someone explain to the mathmatically challanged (me), what "normsinv" in the equation is? and where am I supposed to plug (h) in?

[bisonbison]

For those working in BB/100 and Pokertracker:

A1: Hands
B1: Hand Count from PT
A2: EV/100
B2: BB/100 from PT
A3: SD/100
B3: SD/100 from PT session tab, 'more detail' button.
A4: Confidence Interval
B4: an integer between 0-99 (note: Homer originally used 0-.99)
A5: Upper Bound
B5: =B2-NORMSINV((100-B4)/200)*B3*(1/SQRT(B1/100))
A6: Lower Bound
B6: =B2+NORMSINV((100-B4)/200)*B3*(1/SQRT(B1/100))

[Homer]

[ QUOTE ]
What are the limitations on this formula? I have my numbers in, and things seem to look alright, but my .50/1 limit results seem way off.

Now I just started over at a new site (for a rakeback), and only have 674 hands. I have won 17.54 BB/100 hands, with EV/hr of 10.28 and SD/hr of 11.

Now according to your post I can now say that "I am %90 sure that that I will be winning $5-$15.6 at .5/1" which seems completely absurd. My only thought here is that we need the true SD/hr instead of the statistic, which we can never truly have. How big does our sample size need to get before we can use this formula?

Thanks,
Spicymoose

[/ QUOTE ]

I'm not sure what the limitations are. I'm thinking that once your SD levels off, it should be safe, but I'm not 100% sure.

[Homer]

[ QUOTE ]
will someone explain to the mathmatically challanged (me), what "normsinv" in the equation is? and where am I supposed to plug (h) in?

[/ QUOTE ]

You don't plug number of hands in directly (I don't know why I put an "h" there), you use either sets of 100 hands (if using WR/100 and SD/100) or hours played (if using WR/hr and SD/hr).

normsinv is the inverse normal distribution function. I'm too lazy to explain it further right now, and even if I tried I probably wouldn't do it justice.

-- Homer

[Jay36489]

Interesting, thanks for the post homer. After 13K hands im 95% sure im beating .5/1 between 6.44 and .91 BB/100 (average of 3.67). It seems you have to have an enormous amount of hands to have any certainty. I don't see how you can ever be sure of your win rate because you move up before you get enough hands for mathmatical certainty. I'm already at my trigger point for 1/2.

[Evan]

[ QUOTE ]
I'm thinking that once your SD levels off, it should be safe

[/ QUOTE ]
I think the issue is figuring out when that happens. Or should you just assume that once it stops changing a lot you're money?

[Homer]

[ QUOTE ]
[ QUOTE ]
I'm thinking that once your SD levels off, it should be safe

[/ QUOTE ]
I think the issue is figuring out when that happens. Or should you just assume that once it stops changing a lot you're money?

[/ QUOTE ]

Mason wrote an article once telling people how to compute SD. I believe he said you need 30 sessions. (this is of B&amp;M play). I'll guess that's 5 hrs/session at 35 hands/hr. If I'm right, that comes out to about 5000 hands.

-- Homer

[jasonHoldEm]

Neato. This should get an FAQ link.

J

[One Word Answer]

27,169

[Lost Wages]

Just what we need. Another annoying poster.

Lost Wages

[kdog]

At least his posts are brief. /images/graemlins/smile.gif

[BillC]

To get within 10% of actual win rate you need a sample of about 40,000 hours. To get an estimate to within 20%, you need only 10,000 hours. I am assuming that SD/EV=10 here. If you want to figure it out using your own stats, the 95% confidence formula is

n=((196*SD)/(EV*r))^2

where n=no. of hands needed to estimate EV per hour (=win rate) to within r% of the actual EV. ^2 means "squared".

BillC, Ph.D.

[lacky]

well, I'm not a phd, but I do know there is a big difference between 10000 hands and 10000 hours. So, does n= number of hands or number of hours. If it's hands it seems low, if it's hours, it seem A BIT MUCH! lol

Steve

[r3vbr]

Can someone please send me the excel spreadsheat to wabl@terra.com.br !

My Excel is in another language (portuguese) and the formulas don't work here. But when I downloaded Homer's sheat "Streaks.xls" it worked.

[HesseJam]

You just need to replace NORMSINV and SQRT with the Portuguese names of the functions. You'll find the Portuguese name for NORMSINV in the streak sheet in column J (Amount Won (BB's, hour)). SQRT is "square root". You should find the Portuguese equivalent easily under your function ( fx ) button (look for math&amp;trigonometry).

[ArthurD]

Do I input total hands played in B1, or is it sets of 100 hands played as Homer indicated? This makes a major difference in the outcome of the formula.. I'm using the formulas BisonBison reposted. Great formula, just wanna make sure the valeus I'm seeing are correct.

[Homer]

[ QUOTE ]
Do I input total hands played in B1, or is it sets of 100 hands played as Homer indicated? This makes a major difference in the outcome of the formula.. I'm using the formulas BisonBison reposted. Great formula, just wanna make sure the valeus I'm seeing are correct.

[/ QUOTE ]

If for win rate and standard deviation you are using BB/100, then for total hands you use sets of 100 hands. If you're using BB/hr, then for total hands you use hours played.

-- Homer

[ArthurD]

I have never done this before, but this thread is SO useful that it really needs to get bumped. I really think it anwsers a question that so many ask. Anyways, *BUMP*.
Sorry if this breaks any rules.

-Arthur

[stankybank]

[ QUOTE ]
Introduction

Recently, I've noticed a lot of players asking how many hands they need to play in order to prove that they are a winning player or that they are winning at a certain rate. Usually, the responses received are along to lines of:

- The vague answer -- "You need way more hands than that"

- The random number answer -- "You need at least 100,000,000 hands"

Anyway, I've decided to go over the process of computing confidence intervals, which will allow you to make mathematically valid statements about your true win rate.

Details

In order to compute a confidence interval, you need to locate certain statistics in Poker Tracker, specifically total hands, total table-hours, EV/table-hr (Amount won / Hours played) and SD/hr. All of these statistics are located under Session Notes -&gt; Session Summary, with the exception of standard deviation, which is displayed after a user clicks the "More Detail..." button.

I just started using Poker Tracker again to track my 5/10 SH play, so I'll use those numbers as an example since I was planning to compute the confidence interval for that data anyway.

Hands (h) = 3115
Table-hrs (n) = 32.45
EV/hr (ev) = $39.41
SD/hr (sd) = $182.97

Once you have these statistics, you can use the below formulas to compute the upper bound (u) and lower bound (l) for a given confidence interval (c).

u = ev - normsinv[(1-c)/2]*sd*[1/sqrt(n)]

l = ev + normsinv[(1-c)/2]*sd*[1/sqrt(n)]

For the sample data I set the confidence interval at 95% and with these formulas found that the upper bound is $102 and the lower bound is -$24.

With this information, I can make the following statement:

"I am 95% confident that my true win rate is between -$24 and $102 per table-hr."

-- If I change the confidence interval to...

75%; Lower bound = $2 Upper bound = $76

50%; Lower bound = $18, Upper bound = $61

-- If I keep the confidence interval at 95% and change the hours played to...

100 (9600 hands); Lower bound = $4, Upper Bound = $75

250 (24,000 hands); Lower bound = $17, Upper bound = $62

1000 (96,000 hands); Lower bound = $28, Upper bound = $51

5000 (480,000 hands); Lower bound = $34, Upper bound = $44

10,000 (960,000 hands) Lower bound = $36, Upper bound = $43

From this, you can see that after 3000 hands you can't prove much of anything. It is within the realm of possibility that I have a true win rate of -2.5 BB/hr despite the fact that over 3000 hands I won at a clip of 4 BB/hr. You can also see that even after 500,000 hands, it is not a rarity for a player's true win rate to be 0.5 BB/hr more or less than his results to date.

Creating an Excel spreadsheet

It is quite easy to setup an Excel spreadsheet to compute confidence intervals. Information need only be entered in a few cells, as follows:

Cell A1 - "Hours Played"; Cell B1 - Hours played value (from PT)

Cell A2 - "EV/hr"; Cell B2 - EV/hr value (from PT)

Cell A3 - "SD/hr"; Cell B3 - SD/hr value (from PT)

Cell A4 - "Confidence Interval"; Cell B4 - ".95"

Cell A5 - "Upper Bound"; Cell B5 - <font color="green">=A2-normsinv((1-A4)/2)*A3*(1/sqrt(A1))</font>

Cell A6 - "Lower Bound"; Cell B6 - <font color="green">=A2+normsinv((1-A4)/2)*A3*(1/sqrt(A1))</font>

After you do this, you can mess around with cells A1, A2 and A4 (probably not A3 as your SD/hr converges fairly quickly and isn't likely to change unless your playing style does).

-- Homer

[/ QUOTE ]

[IsaacW]

[ QUOTE ]
What are the limitations on this formula?

[/ QUOTE ]
The formula as shown needs the true SD to be accurate. We do not have that information, but we do have the sample SD (the number that PT reports). When using the sample SD, you need to use something called the t-distribution to find confidence intervals that are wide enough. Using the normal distribution will result in smaller confidence intervals than the true values for small samples.

Information about the t-distribution. (http://mathworld.wolfram.com/Studentst-Distribution.html)

[La Brujita]

Isaac,

I looked at the link but it didn't make a lot of sense to me (I am dumb). Are you saying that if my confidence intervals are say +/- .5bb/100 as determined by PT stats, t-distribution would indicate larger intervals? And generally how much larger?

Thanks in advance.

[Baulucky]

This post is so good that merits a bump.

[Nomad84]

I thought Student's T Distribution only applied when there were a small number of samples. Don't remember the cutoff point, but if you have enough sessions, this shouldn't matter, should it? How many is "enough"? 30? 40? 50?

[mmbt0ne]

I've done quite a few t-distribution problems, and never heard of this "enough" number you speak of. Are you sure you're not confusing something else, like normal approximation of the binomial?

[Mroberts3]

ok, i must be computer illerate. once I have the numbers plugged in, how do i get excel to perform the operation? (for someone whos uses comps as much as I do i cant believe i can not find out how, lol)

[PokerBob]

[ QUOTE ]
ev - normsinv[(1-c)/2]*sd*[1/sqrt(n)]

l = ev + normsinv[(1-c)/2]*sd*[1/sqrt(n)]



[/ QUOTE ]

I'm trying to crunch these numbers. Can someone tell me what "normsinv" is?

[Baulucky]

[ QUOTE ]
I'm trying to crunch these numbers. Can someone tell me what "normsinv" is?

[/ QUOTE ]

An Excel statistic function fully explained in the help section of said program.

[PokerBob]

[ QUOTE ]
[ QUOTE ]
I'm trying to crunch these numbers. Can someone tell me what "normsinv" is?

[/ QUOTE ]

An Excel statistic function fully explained in the help section of said program.

[/ QUOTE ]

So I guess I can't do this with my calculator???

[obsidian]

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
I'm trying to crunch these numbers. Can someone tell me what "normsinv" is?

[/ QUOTE ]

An Excel statistic function fully explained in the help section of said program.

[/ QUOTE ]

So I guess I can't do this with my calculator???

[/ QUOTE ]
Nope. But you could go look it up in a z-score table (http://www.epatric.com/documentation/statistics/z-score_table.html). If your confidence interval is 95%, you take (1-0.95)/2, or 0.025, subtract it from 1 to get .975, and then look for .975 in the table. In this case, it is 1.96.

[obsidian]

I'm not sure if this is new (don't remember seeing it before) but PT also offers SD/100 hands. So would this equation work as well?

BB/100 hands +/- NORMSINV((1-confidence)/2) * SD/100 * 1/SQRT(hands/100)

[mindflayer]

A monkey in the zoo thanks YOU! for this thread.

[Josh.]

Homer puts my stat posts to shame /images/graemlins/frown.gif

[ryanflies]

btw, I uploaded the spreadsheet to an htm. http://e.sixbit.org/howmanyhands.htm

according to my numbers at $1/2:

hours (PT) 7.38
wr/hr (PT) in $ 12.33
sd/hr (PT) in $ 14.1601
confidence (PT) 0.95

I'm 95% confident that my wr is between $22.54612576/hr and $2.113874238/hr.

That doesn't seem right with such a small sample. Is it?

[Dan (Theman)]

I think that is because your SD is no where near accurate after 7 hours. You have been running well with a low SD so far. Your SD will probably rise.

[Webster]

Very interesting post - a little confused over the "set's of 100" or actual hands.

If I have 31,214 hands played is that 312 in column B2 or should I put 31214. Can't wait to get home and plug in some numbers.

[astrodon]

[ QUOTE ]
I'm not sure what the limitations are. I'm thinking that once your SD levels off, it should be safe, but I'm not 100% sure.

[/ QUOTE ]

But certainty is not what this is about; confidence is and 95% is usually good enough... /images/graemlins/wink.gif

[nofutures]

For the germans out there. The german excel -forumla goes:

=A2-STANDNORMINV((1-A4)/2)*A3*(1/WURZEL(B1))

[$hort$tacked]

awesome post

[Adde]

I was looking for this excelent post by Homer, and thought I might as well bump it.

Adde

[Iq75]

[ QUOTE ]
For those working in BB/100 and Pokertracker:

A1: Hands
B1: Hand Count from PT
A2: EV/100
B2: BB/100 from PT
A3: SD/100
B3: SD/100 from PT session tab, 'more detail' button.
A4: Confidence Interval
B4: an integer between 0-99 (note: Homer originally used 0-.99)
A5: Upper Bound
B5: =B2-NORMSINV((100-B4)/200)*B3*(1/SQRT(B1/100))
A6: Lower Bound
B6: =B2+NORMSINV((100-B4)/200)*B3*(1/SQRT(B1/100))

[/ QUOTE ]

for the Finns:

B5 =B2-NORM.JAKAUMA.NORMIT.KÄÄNT((100-B4)/200)*B3*(1/NELIÖJUURI(B1/100))

B6 =B2+NORM.JAKAUMA.NORMIT.KÄÄNT((100-B4)/200)*B3*(1/NELIÖJUURI(B1/100))

[rjacobs003]

Question on this. Would the SD/100 then effectviely be the variance you experience in your win-rate? Are there bechmarks for where this number should settle?

[uDevil]

[ QUOTE ]
Question on this. Would the SD/100 then effectviely be the variance you experience in your win-rate? Are there bechmarks for where this number should settle?

[/ QUOTE ]

Variance= SD^2

For limit Hold'em, a typical SD~16 BB/100. For other games, try searching the appropriate strategy forums for stat posts.

[3N1GM4]

I just plugged some numbers in from a bout of micro stakes games:

Hours: 137.58
EV/hr: $1.85
SD/hr: $8.6349

With these stats it's saying I have a 98.7% certainty of being a winning player at this level (i.e. Lower bound &gt;= 0) This seems odd for such a small sample... is it right?

(Edit: NLHE, btw)

[3N1GM4]

Anyone?

[Emperor]

[ QUOTE ]
I was looking for this excelent post by Homer, and thought I might as well bump it.

Adde

[/ QUOTE ]

Me Too!