Two Plus Two Newer Archives  

Go Back   Two Plus Two Newer Archives > Limit Texas Hold'em > High Stakes Limit

Reply
 
Thread Tools Display Modes
  #321  
Old 05-11-2005, 09:29 PM
elindauer elindauer is offline
Senior Member
 
Join Date: Jun 2003
Location: analyzing hand ranges
Posts: 2,966
Default Confidence intervals are not accurate here

[ QUOTE ]
With an SD of 30BB/100 hands ... interval is 2.8BB/100 at a 99.7% confidence... So the math tells us that he is likely a winner... it'll take more than some bad beat stories and a few hand histories before I'll write DERB of as a fluke.

[/ QUOTE ]

Hi rigolleto,

Your math is off here, but it's subtle why it's wrong. The problem is the way you've selected the "random variable" to analyze.

Basically, there are hundreds, maybe thousands of guys with stats like DERB. All of these random variables have been sampled over and over, and then the very best performing one has been chosen and singled out. There is a huge selection bias here, so that it is extremely likely that DERB is in fact way on the high side of the variance. Since the confidence interval calculation you are making assumes it's equally likely he is on the high side as on the low side, it's invalid.

To give you a clearer picture, imagine you flipped a large number of pennies ten thousand times each. You record the results. Then you pick out the penny that came up heads most often and run a confidence interval analysis on it. You'll find that this analysis suggests it's extremely likely that this penny comes up heads more often than tails. Maybe it'll suggest that this penny is 99.7% likely to be weighted toward heads, to not be a fair coin. Obviously, this is not the case.

To make another analogy, it would be like taking any random bad player, and then throwing out 10% of the hands in which he did the worst. Oh look, he's a huge winner now. Yeah, of course.

In light of this, anecdotal evidence suggesting that DERB is bad is all you should need to be confident he is not a winner and is simply the luckiest player at party. This sounds unlikely until you remember how we chose this guy... we searched the database for the most ridiculous stats / win combination. Of course, this is likely to correspond to the luckiest guy.

To get a better idea of how DERB's true win rate, lump all the players with similar stats to his together and run a confidence interval analysis on the results. To get an even better idea, track his NEXT 100,000 hands or whatever, now that you've singled him out and do a confidence interval analysis on those hands. That's his true win rate.


Good luck.
Eric
Reply With Quote
  #322  
Old 05-11-2005, 09:43 PM
Glenn Glenn is offline
Senior Member
 
Join Date: Sep 2002
Location: New Jersey
Posts: 1,258
Default Re: WHAT IS DERBS SCREENAME ON PP???

I hate this thread so much I don't know why I'm posting in it but anyway...

Without writing a novel, Justin A is correct. If you simply take his win rate and SD you are ignoring selection bias. To get the best approximation, you must use all available information. This not only includes win rate, SD, and a Z-table, but also includes the more fuzzy idea that this player is being examined because of his win rate in the first place, and the anecdotal evidence that he plays like players who lose. If you have 1,000 players who play the same, one of them will have a higher win rate than the rest, and this will be significantly higher than the average. If you look at the sample of 1,000 and choose the highest win rate and then try to reverse engineer his true win rate using a confidence interval, you are going to get a very wrong answer because you are ignoring the information that he plays the same as the other 1,000 players. Of course this is not a direct analogy because no players play exactly the same, but you are making the same mistake as someone who chooses a mutual fund that showed a huge profit in the past year. If you put enough monkeys in front of enough typewriters, eventually one will type HPFAP. If, after this, you select the one who does and claim he is Mason Malmuth, you are of course wrong. Or are you? [img]/images/graemlins/smile.gif[/img]
Reply With Quote
  #323  
Old 05-11-2005, 09:45 PM
Justin A Justin A is offline
Senior Member
 
Join Date: May 2004
Location: Clark County
Posts: 6,340
Default Re: WHAT IS DERBS SCREENAME ON PP???

[ QUOTE ]
[ QUOTE ]
We're doing the same here with DERB. His stats were chosen because of the bizarre results, therefore we cannot use confidence intervals to figure out whether or not he is a winning player. I hope all that made sense.

[/ QUOTE ]

I have to disagree. Your objection would be true if we had say 1,000,000 hands with DERB and picked a 100K series out of them because this series was bizarre. But the fact that we chose to isolate all the hands where DERB was at the table doesn't bias HIS stats.

You analogy is flawed because you are assuming that each player can be represented by the same coin. A more true analogy is to make 1000 cointosses with 1000 coins and the result is say 500,152/499,848. You record all tosses and what coin they where made with. Now you isolate your sample to a specific coin and find that the result for this coin is 611/389. You now have good reason to suspect that this particular coin is different from the others because the 1000 tosses with this coin can statistically be treated as a seperate event.

Let me try my own analogy: let say we randomly sample 4000 people and ask them who they would vote for as president Andyfox or Ed Miller and the result came back 50/50. Now we pick a subset, say ages 18-30, and for this subset the result is 30/70. Just because we picked a subset it doesn't mean that the sample is not random anymore; it is a random sample within the universe of 18-30 year olds. Just as hands with DERB in them is a random sample within the universe of hands DERB participated in.

[/ QUOTE ]

Please see elindauer's post, he explains it better than I do.

Your analogy is flawed because you have chosen the 18-30 group before seeing the results. This is not a selection bias.
Reply With Quote
  #324  
Old 05-11-2005, 09:46 PM
elindauer elindauer is offline
Senior Member
 
Join Date: Jun 2003
Location: analyzing hand ranges
Posts: 2,966
Default Re: WHAT IS DERBS SCREENAME ON PP???

Hi rigolleto,

Justin is right on here. Think more about what he's saying. I'll explain where your logic is off below.

[ QUOTE ]
[ QUOTE ]
We're doing the same here with DERB. His stats were chosen because of the bizarre results, therefore we cannot use confidence intervals to figure out whether or not he is a winning player. I hope all that made sense.

[/ QUOTE ]

I have to disagree. Your objection would be true if we had say 1,000,000 hands with DERB and picked a 100K series out of them because this series was bizarre. But the fact that we chose to isolate all the hands where DERB was at the table doesn't bias HIS stats.

[/ QUOTE ]

You're right that we haven't biased DERB's stats, but we are biased in selecting DERB to begin with. If DERB were a losing player, the way so many players who play like him are losers, than we wouldn't be talking about him. He is only interesting because his results are so good. This is why and how he was chosen. It's not like we said, let's pull out a random guy with 30/18 stats and analyze his play... oh look, DERB. No, it's, let's look for a statistical anomaly among the thousands of players I've tracked... hey, check out this guy!

[ QUOTE ]
You analogy is flawed because you are assuming that each player can be represented by the same coin. A more true analogy is to make 1000 cointosses with 1000 coins and the result is say 500,152/499,848. You record all tosses and what coin they where made with. Now you isolate your sample to a specific coin and find that the result for this coin is 611/389. You now have good reason to suspect that this particular coin is different from the others because the 1000 tosses with this coin can statistically be treated as a seperate event.

[/ QUOTE ]

This is only true if you choose the coin randomly. If you choose the coin precisely because it's the one with the most skewed stats, then you've biased the game and confidence intervals don't apply.

[ QUOTE ]
Let me try my own analogy: let say we randomly sample 4000 people and ask them who they would vote for as president Andyfox or Ed Miller and the result came back 50/50. Now we pick a subset, say ages 18-30, and for this subset the result is 30/70. Just because we picked a subset it doesn't mean that the sample is not random anymore; it is a random sample within the universe of 18-30 year olds. Just as hands with DERB in them is a random sample within the universe of hands DERB participated in.

[/ QUOTE ]

Again, your reasoning is accurate if you chose 18-30 randomly, and not because the results in that age group were abnormal.

Finally, let's say you flip 1 coin lots of times. You expect that a confidence interval will have 50/50 as well within the expected value for the coin. However, if you repeat the experiment many many times, eventually you are going to find a coin that comes up 5 standard deviations out. This isn't surprising, it's expected, and it doesn't make it 98% likely that the one lucky coin is biased. It's just a function of the large number of random variables. To get a feel for how likely it is that we would find someone so many standard devs out, we need to find out how many players are in the database that this guy was taken from. I bet the answer is... LOTS.

Good luck.
Eric

PS. It's also interesting to note that the average player with a decent number of hands in a database like this is going to be on the high side of the variance. You don't see guys with DERB's stats that are monster losers simply because a losing player that goes on a big downswing is likely to just quit. So guys with small numbers of hands will tend to be on the low side of the variance, while guys with large numbers of hands will be on the high side. Does that means playing lots of hands makes you lucky? No. It's just a biased sample.
Reply With Quote
  #325  
Old 05-11-2005, 09:54 PM
gol4pro gol4pro is offline
Senior Member
 
Join Date: Apr 2005
Posts: 2,087
Default Re: DERB

Can someone PM me this guy's Screen name for PP? I'm really interested in watching the guy even though I don't usually play at PP.

I took statistics back in college for a semester, and all this stuff about Z values, standard deviations, confidence intervals, and other stuff is bringing back some memories.

Either he's a cheater, or he's amazing. It's just statistically impossible to run THAT well over 150k hands if you're a bad player.
Reply With Quote
  #326  
Old 05-11-2005, 10:00 PM
shmahappens shmahappens is offline
Senior Member
 
Join Date: Nov 2004
Posts: 442
Default Re: DERB

I feel like eventually the amount of views of this post will become more than the hands in my database...
Just to add another worthless concept to a rediculous thread.
Reply With Quote
  #327  
Old 05-11-2005, 10:09 PM
fnord_too fnord_too is offline
Senior Member
 
Join Date: May 2004
Location: February made me shiver
Posts: 9,200
Default Re: Confidence intervals are not accurate here

[ QUOTE ]


In light of this, anecdotal evidence suggesting that DERB is bad is all you should need to be confident he is not a winner and is simply the luckiest player at party. This sounds unlikely until you remember how we chose this guy... we searched the database for the most ridiculous stats / win combination. Of course, this is likely to correspond to the luckiest guy.



[/ QUOTE ]

Again, you can use Baysean Statistical Inference to blend expert oppinion and data.

Also, you can quantify the liklihood that this is the result of survivor bias.

I think assuming this guy is a lottery winner based on intuition is not very rigorous, to say the least. Greater minds than ours thought and convinced the world that heavier objects fell faster than lighter ones because it was intuitively obvious.
Reply With Quote
  #328  
Old 05-11-2005, 10:41 PM
steveyz steveyz is offline
Senior Member
 
Join Date: Oct 2003
Posts: 1,601
Default Re: Confidence intervals are not accurate here

[ QUOTE ]
[ QUOTE ]
With an SD of 30BB/100 hands ... interval is 2.8BB/100 at a 99.7% confidence... So the math tells us that he is likely a winner... it'll take more than some bad beat stories and a few hand histories before I'll write DERB of as a fluke.

[/ QUOTE ]

Hi rigolleto,

Your math is off here, but it's subtle why it's wrong. The problem is the way you've selected the "random variable" to analyze.

Basically, there are hundreds, maybe thousands of guys with stats like DERB. All of these random variables have been sampled over and over, and then the very best performing one has been chosen and singled out. There is a huge selection bias here, so that it is extremely likely that DERB is in fact way on the high side of the variance. Since the confidence interval calculation you are making assumes it's equally likely he is on the high side as on the low side, it's invalid.

To give you a clearer picture, imagine you flipped a large number of pennies ten thousand times each. You record the results. Then you pick out the penny that came up heads most often and run a confidence interval analysis on it. You'll find that this analysis suggests it's extremely likely that this penny comes up heads more often than tails. Maybe it'll suggest that this penny is 99.7% likely to be weighted toward heads, to not be a fair coin. Obviously, this is not the case.

To make another analogy, it would be like taking any random bad player, and then throwing out 10% of the hands in which he did the worst. Oh look, he's a huge winner now. Yeah, of course.

In light of this, anecdotal evidence suggesting that DERB is bad is all you should need to be confident he is not a winner and is simply the luckiest player at party. This sounds unlikely until you remember how we chose this guy... we searched the database for the most ridiculous stats / win combination. Of course, this is likely to correspond to the luckiest guy.

To get a better idea of how DERB's true win rate, lump all the players with similar stats to his together and run a confidence interval analysis on the results. To get an even better idea, track his NEXT 100,000 hands or whatever, now that you've singled him out and do a confidence interval analysis on those hands. That's his true win rate.


Good luck.
Eric

[/ QUOTE ]

Great explanation!

Another thing to note is that the confidence interval is based on the assumption that BB/100 winrate follows a normal distribution, which isn't necessarily a good assumption.
Reply With Quote
  #329  
Old 05-11-2005, 11:17 PM
rigoletto rigoletto is offline
Senior Member
 
Join Date: Sep 2002
Location: Denmark
Posts: 2,345
Default Re: WHAT IS DERBS SCREENAME ON PP???

Hi Eric

Thank you for taking the time. It does look like I'm wrong on this, but if I'm learning that's all good. But I still need to wrap my brain around something:

Your assertions would also seem to imply that DERB can not run a confidence interval on his own stats. I mean, we have determined that confidence intervals doesn't apply. Let's for arguments sake assume that the 100K hands we are talking about are all the hands DERB has ever played; how can the confidence interval then apply to one analysis and not another.

Aren't we confusing a (very small) sample of players (only DERB) with the sample of hands. Granted there is a relationsship between his sample of hands and the key numbers we chose him from, but does this really mean that a sample of 100K hands are rendered meaningless just because we went looking for a LAG that is a winning player?

Let's imagine I where to look up 30/18 (or there about) players with a good winrate in a huge database and I find 100 of them. Now running a confidence interval on this sample of 100 players and their 4 key stats will tell me that they are likely winners because that's why I picked them (the confidence interval is meaningless like in your coin example). But when I look closer I find that 99 of these players stat's are based on less than 100 hands while one is based on 100K hands. Doesn't this make a difference! Intuition tells me that the key figures from the guy with 100K hands are more reliable than from the 100 hand guys. It also seems to me that doing a confidence interval on the 100 hand guys will tell us that they could be anywhere from big loosers to big winners, while the confidence interval on the 100K hands points to a winner.

I can't really find a good coin analogy for this one.
Reply With Quote
  #330  
Old 05-11-2005, 11:20 PM
sxz18 sxz18 is offline
Senior Member
 
Join Date: Jun 2004
Posts: 212
Default Just curious if this has anything to do with DERB

I might be way off here, but if a top tier professional...say Phil Ivey were to play 30/60 on PartyPoker, would he demolish the game? Or is the difference between the best 30/60 players on PartyPoker and Phil Ivey a very slim margin? If, by chance, DERB turns out to be a top tier professional, is it hard to believe he can play so many hands and still win? I'm just throwing out another possibility.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 03:46 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.