Probability tests for pokertracker stats

[KotOD]

Hi forum. I PM'ed BruceZ asking if this was appropriate and he gave me the affirmative, so here goes. The Zoo needs some expert math help for some debunking or bunking, I guess.

In this thread:
http://forumserver.twoplustwo.com/showfl...=0#Post10298242
The OP found some strange behavior in some opponents, exploited it, did some digging and then accused the opponents of beings bots based on these numbers

Clickable thumb

The Zoo then tore them apart and noticed the ridiculous similaries in the numbers, especially numbers that would be hard to duplicate between four different players like river bet %.

575/2497 = 0.230276332
216/961 = 0.224765869
538/2403 = 0.223886808
411/1836 = 0.223856209

The main player in these accounts came forward and claimed that the four players were a tightly coached and coordinated team that had a set script and consulted each other on all tough decisions and that's why the numbers were so similar.

Of course the counter was that even with that setup, humans couldn't dulpicate results so closely, especially on postflop play because of all of the variables involved.

So my question is this, working with the numbers in the picture, are there any tests that can be done to draw a conclusion one way or another as to whether the numbers are acutally extremely closely correlated (there is much dispute about the VPIP numbers, less about the river bet % numbers)? If there are, are there any tests that can determine the probability of numbers that bunch like this being created by humans?

If you have any questions, I'll try to help as much as possible.

Thanks

[T50_Omaha8]

Well, I worked some of it out, and I'm too drunk to say officially. Maybe I'll be sober enough to show you some tests tomorrow, but right now I'll say the following:

It's not bloody likely.

[BlueBear]

These statistics alone are not enough to "prove" that they are from the same player. You can't use a "simple" hypothesis testing procedure like a chi-squared test, and that won't indicate anything. A statistician examining the likelihood they are from the same player needs to have to other peoples stats and see how they behave. How much do these stat differ across 100k+ hands tight-nitty players?

There is a second issue with that there may be some correlation in some of the variables. For example, the "flop aggression", "bet flop", "raise flop" and "c/r flop" are connected to each other and a player with a high "flop aggression" will reasonably have high stats in the other 3 categories. An analysis of the likelihood they are the same player must have any correlation effect isolated. How these correlated variables behave in reality can be examined by studying the correlation of other 100k+ hands players.

Extreme care must be taken to conduct any probability tests and it is too easy to use a poorly-designed statistical test that does not consider the matters I outlined above. This is a formidable full-time task and should be undertaken by somebody with a solid postgraduate education in statistics (or having similar experience). I am not defending anybody here, but it is too easy to get carried away and it is easy to "prove" they are from the same player, but such test would lack mathematical rigority.

I'm not a statistician/mathematician but these are just my views.

[2OuterJitsu]

Cliff notes:
Out of 79,277 players and 44,439,607 total hands, not a single pair of players with >5000 hands, have any matching 4 or more stats at any time since Jan 1 2007 in my database (not FTP).


I happen to be 4 weeks into a personal project of my own. I’m a DBA (8 years) so I can write a query or two. I have a summary of statistics of 44,439,607 hands/79,277 players. This database is a summary of NL$200 6 max going back to Jan 1. It includes only players with >5000 hands. It’s a summary so I can’t drill down to hours played or positions or starting hands, etc.

For the record, I’m no statistician. I’m no math wiz. Hell I’m no poker phenom either (the reason for this project). I’m a pretty good DBA though, so after reading this thread, I wanted to see how many players could (do in this case) have so many similar statistics. Here’s what I’ve found:

VPIP: 34 groups > 3
VPIP & PFR: 115 groups > 3
VPIP & PFR & WtSD: 27 groups > 3
VPIP & PFR & WtSD &W$SD: 0 groups

????

There are no groups of players with the same ANY 4 STATS*! These guys have 9! Again I’m no statistician, but IMO including samples as small as 5000 hands should increase the possibility of 4 players have 4+ matching stats. 5000 hands is squat; winning and loosing players have similar stats with such a small sample size. Going back to Jan not even 1 group???

These guys have managed to do it. 9 matching stats, they all know each other, and they all play at the same time, in the same room???

Someone with more math sense than me, flame my theory please. I checked: VP$IP, PFR, TotA, FlopA, TurnA, RiverA, WtSD, and W$SD.



*After reading more posts and opinions I check only the Agression stats and there are 23 groups >= 3. These are the only 4 stats with groups. There are definately no other 4 or more matching stats.

[neverforgetlol]

You aren't interested in correlation really, you want to do proportion tests, at least first, comparing whatever you'd like, vpip, pfr, etc. The first test I did in the big thread compared vpip of two of the suspect bots, with a test statistic of -.00005, meaning we are going to not reject Ho: p1=p2 at pretty much any significance level. This is pretty good evidence in favor of them being bots.

[imfatandugly]

why cant this be 1 guy playing on three different screen names. what is the probability that if you were to play this many hands, split this into fourths that they would be this close?

[Siegmund]

That's the basic limitation of this type of test.

What the statistics say is that the 4 players have very nearly identical playing styles. What the statistics cannot do is tell us whether they are 4 bots, or 4 players following a recipe precisely, or 1 player with 4 accounts.