A quick note on stats and convergence

[OrigamiSensei]

My mini-rant is based on a hand post that came out in the last week that I can no longer find but that I felt the need to comment on. In the hand post the heads-up stats were provided for a player with 46 or so hands of data, during which time they had a PFR stat of either 2% or 4%, can't remember which. At some point I remember reading at least one reply that basically assumed the PFR number was accurate based on this very small number of trials. Now for people versed in statistics and the long run this is obviously a large leap of faith, but for people new to the game and new to tools such as PAHUD and GameTime+, particularly micros players this may not be so obvious. So if the experienced micro'ers and the folks who understand math and statistics will bear with me on this one I think it's worth reminding newbs to be careful about trusting these numbers overmuch with small sample sizes.

In the thread I can no longer find the oppnent being analyzed had raised either once or twice in the space of around 46 hands, I don't remember the exact numbers. But for our purposes it will be sufficient to examine the possibility that this seeming rock is actually a normal TAGgy 9% PFR type on a cold run of cards. By going back to my old probability textbook I was able to track down the formula for the binomial distribution. This allows us to calculate the probability that a particular event will occur n times in y trials, with p representing the probability of that particular event:

f(y) = (n select y) p^y*(1-p)^(n-y) where (n select y) is n!/(y!*(n-y)!)

If we fill in the numbers where 9% (p=.09) is the actual pre-flop raise probability for this player, n is a span of 50 hands and y is either 1 or 2 depending on whether the player raised once or twice (thus looking like a 2% or 4% PFR player) we get the following:

A 9% PFR player has a 10.7% probability he will raise only twice in a given 50 hand span and a 4.4% probability he will raise only once. Adding the two together is over 15%, or close to 1 in 6 times.

Now, are we ready to really assume that this player only raises 2% or 4% of the time absent any other evidence? We shouldn't be quite ready to yet. Now, of course if we see the guy open-limp and show down hands like AJ from the cutoff then that's pretty strong evidence the heads-up display is accurate. But absent other evidence, particularly if you've been multi-tabling and not taking mental note of things like that you can't rely so completely on the number.

Like I said, obvious to the more experienced but hopefully it's a thought of some use to newer players. Sorry I lack the mathy knowledge to fully calculate confidence intervals and such but I didn't want to turn this into a thread for SMP or the Probability forum, just wanted to throw out something to consider.

[ckj]

Thanks for that OrigamiSensei. It's definitely something to keep in mind. It's also important to note that the stats are just a guideline and even tight players will do completely insane things from time to time.

[Mitke]

I believe you looked for this thread: AQo on the Button, am I right?

The info you provide is good and correct. I tend to fall to this trap occasionally when multitabling.

However, for those of us that might have short attention spans a more useful info would be e.g. that after how many hands does vpip, pfr, stats are accurate enough to be X +1 or X-1 % with say 95% reliability. Or that how reliable these are after e.g. 50, 100, 500 hands. I'm not asking you to calculate these nor am I competent to do that myself either.

The short-handed limit book of Borer, Mak and Tanenbaum quotes 200 hands for vpip and pfr to be "reliable" to some extent.

I'm pretty sure there's a thread here somewhere that discusses these. I don't have time to search for these now. If some of the older hands happens to remember a few good threads I'd be delighted if you take the time to post links to them here.

[neurotiq]

Great post, OrigamiSensei. Thank you. [img]/images/graemlins/smile.gif[/img]

[Smurph64]

In my stats I notice that convergence happens usually after any grab of 2000 hands within 2%.

[maverickai]

I've been playing against some bots on UB, and I've collected approx 2000+ hands. Since they would be using the same logic and play strategy, I only start to realise that their stats converge at about 1000+ hands.

Alright, time to go rip the bots apart... [img]/images/graemlins/smile.gif[/img]

[OrigamiSensei]

[ QUOTE ]
I believe you looked for this thread: AQo on the Button, am I right?

[/ QUOTE ]
Yeah, that's the one and obviously other people were clearly making the point about sample size before this post by me. I simply wanted to show that a situation like this is not unusual with a little relatively simple probability formula. Thanks for posting the link.

I will note that VP$IP converges for most players well before just about any other stat and I will make use of it earlier as a consequence. Especially for players who are involved in more hands you get more information on what they are willing to play.

[TimovieMan]

This makes a great addition to the topic I started about stat-reads vs. real reads.

But since stat-reads are not accurate until you get a rather large hand sample on your opponent, then I think this also means that a large majority of all the hands that get posted here simply don’t offer enough info on the opponents. Most of the time we have to make do with a stat read on 50 hands or less, seldom do you see a real read.
However that does add to the value of the replies since you get more replies discussing standard play in any given situation, and even several ifs regarding the different styles your opponent could be playing (TAG / LAG / fish / maniac / rock / weak-tight / etc.), so having unreliable stats doesn’t matter all that much.
It DOES make a big difference on the specific hand that was played, but since we don’t post results, it’s all about the most +EV decisions…
I’m not trying to make a point here, it’s just a random thought of mine…


Regarding the matter at hand: for the non-mathematicians amongst us (or the ones too lazy to do their own research):

As of how many hands (more or less) does the VP$IP-stat become reliable?
And the PFR?
AF?
%WtSD?
W$SD?
Other often used stats?

[bellatrix]

[ QUOTE ]
In my stats I notice that convergence happens usually after any grab of 2000 hands within 2%.

[/ QUOTE ]

Convergence of which stat? You do agree that some converge much more quickly than others. After 2k hands the VPIP accuracy will usually be better than 2%, but e.g winrate will not be even CLOSE to accurate (still +/-100% well possible). Some stats converge quicker than others and just throwing a random number out there (50,2000) without saying why, is not the way to go.

[Smurph64]

pick a stat and it usually comes into alignment after 2k hands, except for win rate.

VPIP, PFR, Aggro, WSD, attempt to steal, are the ones I use to determine player read accuracy.