Winners, losers and variance

[tsearcher]

The math involved here is over my head, but there is a pretty nice example of winning a lot of hands just by pure luck. You can check here: http://www.cs.ualberta.ca/~games/poker/ and more specifically here: http://www.cs.ualberta.ca/~darse/Pap...ings-phd.html.

A quick synopsis, the guys at University of Alberta set up two bots to play a HU LHE match. One bot is always call and the other is always fold.

I am now quoting from "Darse Billings, Ph.D. dissertation, September 2006."

"There is never a fold, so every betting round is involved in every game ... The final pot size is always 14sb, and the net outcome for one player is either +7 sb, -7sb, or zero. ... The long term expected value for each player is exactly zero..."

They ran the match and after 100,000 hands, one of the players was up 2500 sb. They ran it a second time and a player was up 3000 sb after 100,000 hands.

[GiantBuddha]

Yeah, I've gone a few hundred hands in limit without winning a single pot (I did get to chop in the blinds twice), so I'm gonna say it takes a bit longer than that to converge. Seriously, though, I used to think it didn't take long to find out if you were a winning player or not, because I was a winning player and I never had any long losing streaks. The first time I suffered from a slump I thought it was because I'd forgotten how to play well. But the truth of the matter is, sometimes you can't buy a pot. In a series of independent events, anything can happen. Rosencratz and Gildenstern anyone? I think it's easy to watch someone play and see if they might be a winner in the long run or not, but just looking at results tells little over 200-1000 hands.

[uDevil]

[ QUOTE ]
what's possible and what's probable are two different things.

[/ QUOTE ]

Hmmm. The webpage needs a little updating, but perhaps you'd like to play with this a little:

http://www.castrovalva.com/~la/win.htm

[_D&L_]

If the standard deviation after 100 hands is 15BB, why can't we say that a person who is either +- 30BB from zero, is either playing with a winning or losing strategy with more than 95% certainty (after all thats two standard deviations). I don't follow your math, why does adding more hands create your gigantic standard deviation of +- 150 (300 / 2 )

Also, I don't know what your confidence interval proves. Its setup for a meaningless category of people - people subjectively believed to be "solid winners." If i make more than 500+ i prove i'm not a solid winner, but an exceptional winner. If i make less, i may prove i'm not a solid winner, but it doesn't show i'm not a winner.

These categories don't seem very helpful to me. I think people want to know whether their strategy is winning or losing, not if it is a statistically significant deviation from a subjectively determined group of players knighted "solid winners" by a survey taker.




[/ QUOTE ]

[pzhon]

Oh, I see you caught your earlier post in time. I had typed out an informative but not very friendly response, but the server said I couldn't post a response because your message had been deleted. I'm pleased to see your new response is more reasonable.

[ QUOTE ]
If the standard deviation after 100 hands is 15BB, why can't we say that a person who is either +- 30BB from zero, is either playing with a winning or losing strategy with more than 95% certainty (after all thats two standard deviations).

[/ QUOTE ]
You can, but that would be an abuse of statistics, and it's not a powerful test. ("Power" is a technical term in statistics, by the way.)

It's quite uncommon for someone to be ahead 30 BB or behind 30 BB over 100 prespecified hands. If it happens in a test, you do have strong evidence against the hypothesis that the player has a win rate of 0 BB/100, with a standard deviation of 15 BB. However, it would not greatly favor the hypothesis that the player wins 2 BB/100 over losing 2 BB/100. You would mainly have evidence that your standard deviation estimate was too low. Even if you were 100% certain of your standard deviation estimate, you wouldn't have much information refining the prior distribution, which is a better goal than being able to reject the null hypothesis.

[ QUOTE ]
I don't follow your math, why does adding more hands create your gigantic standard deviation of +- 150 (300 / 2 )

[/ QUOTE ]
The standard deviation scales as the square root of the humber of trials. A standard deviation of 15 after 100 hands means the standard deviation is 150 = 15*sqrt(100) after 10,000 = 100*100 hands.

[ QUOTE ]
Also, I don't know what your confidence interval proves. Its setup for a meaningless category of people - people subjectively believed to be "solid winners."

[/ QUOTE ]
The example I gave showed that someone who is assumed to be a solid winner (and I'm simply telling you that 2 BB/100 is a good, sustainable win rate in many games according to my experience and the experience of the limit forums here) can't be very confident of coming out ahead after 10,000 hands. Breaking even (or losing) would not be 2 standard deviations below average.

I'll admit that I used "confidence interval" in a nonstandard (probability) fashion instead of the usual statistical manner. The usual meaning is an interval around an observation, while I referred to other intervals of the same width.

[eigenvalue]

This example proofs just the opposite of what you meant: We can see 2 things here:
(1) If there would be a casino rake involved into that game, both players would be loosers, even the one with more luck. That leads us to the assumption that someone who is a looser over 100.000 hands is not good enough to beat the game under the given conditions.
(2) If someone is a winner including casino rake into the game, it seems that he has to win that much money that it can't come from pure luck alone!

[tsearcher]

I guess it depends on the rake structure. But I've never seen a rake that is more than 100% of the pot. How could the winning player be losing to the rake?

[_D&L_]

Col 1 - number of hands
Col 2 - standard deviation, given 15BB is SD for 100 hands in limit poker, of a 2BB+ / 100 hand player.

Col 3 - Expected mean of a 2bb / 100 hand player.

Col 4 - 95% certainty you make less than 2BB per hand, if you have lost this amount, over the corresponding number of hands

Col 5 - 99.7% certainty you make less than 2BB per hand, if you have lost this amount, over the corresponding number of hands

Some things to note: Its technically possible that maybe 3 in 1000 players who truly average 2BB per hand, may not see a profit until about 13,000 hands.

But its also true you can tell fairly early on, with a high degree of certainty if your losing. For instance, if your down 74BB after 400 hands, its 99.7% certain your not a so-called "solid winner" and making less than 2BB per hand. If your down 82BB after 400 hands (74BB - 400/100 * 2) its just as certain that your making negative BB per hand.


100 15.00 200 -26 -41
200 21.21 400 -34 -56
300 25.98 600 -40 -66
400 30.00 800 -44 -74
500 33.54 1000 -47 -81
600 36.74 1200 -49 -86
700 39.69 1400 -51 -91
800 42.43 1600 -53 -95
900 45.00 1800 -54 -99
1000 47.43 2000 -55 -102
2000 67.08 4000 -54 -121
3000 82.16 6000 -44 -126
4000 94.87 8000 -30 -125
5000 106.07 10000 -12 -118
6000 116.19 12000 8 -109
7000 125.50 14000 29 -96
8000 134.16 16000 52 -82
9000 142.30 18000 75 -67
10000 150.00 20000 100 -50
20000 212.13 40000 376 164
30000 259.81 60000 680 421
40000 300.00 80000 1000 700
50000 335.41 100000 1329 994
60000 367.42 120000 1665 1298
70000 396.86 140000 2006 1609
80000 424.26 160000 2351 1927
90000 450.00 180000 2700 2250
100000 474.34 200000 3051 2577

---- Edit -----

I don't really believe this is the right approach though. Measuring variance from a group of so-call solid winners, introduces a range of players. Even assuming they all have +2BB win rates, we are mixing the guy with the +2BB win rate with the guy who has a +3BB, +5BB, or +12BB, etc. win rate.

It may be that each of those players has a low variance with respect to his particular games, but mix them together, and ta'da high variance. Thus, even though I'm using the 15BB/100 hand variance suggested by the previous poster, its not going to be the best method.

The best method requires analyzing the variance of your individual games...unfortunately that will vary from player to player. But i would think that the variance of one player employing a similar strategy game after game, is going to be lower than a motley crew of "winning" players. Thus, if you fall below the thresholds listed above, you will probably fall below the same threshold if you were to analyze the variance of your specific returns.

Thus, I think the above thresholds "err on the side of caution" in declaring players losers.

[Abbaddabba]

If we start talking about people with unrealistically high winrates, then your conclusions are for sure valid. A 5BB/100 winner should almost never have a few thousand hand stretch where they are down.

But winrates like that are unattainable online these days, unless you go down to the penny tables.

Even 2BB/100 is unattainable for most people who seem to THINK that they're 2BB/100 winners.


Winrates tend to be very low relative to standard deviations. Especially for mid to high stakes players.

If you're dealing with winrates that are around, if not below 1BB/100 with a standard deviation exceeding 20BB/100, any given session means a whole lot less.

[pzhon]

[ QUOTE ]
Its technically possible that maybe 3 in 1000 players who truly average 2BB per hand, may not see a profit until about 13,000 hands.

[/ QUOTE ]
That's highly misleading. You can say that you saw a profit within the first 100 hands if you win a few pots and then go on a 200 BB 20k hand downswing.

After 13,000 hands, breaking even is 1.52 standard deviations below the mean, which means that if you specify 13,000 hands ahead of time, not 3 but 64 players will be behind. Since 64 players in 1000 are behind at the 13000 mark exactly, more players will be behind at some point after than 10,000 hand mark, or at some point between 10,000 hands and 15,000 hands.

Suppose someone says, "WTF? I'm behind 50 BB after 12,600 hands this month!" Do you think he planned to play up to 12,600 hands, quitting when he first made a profit? Did he plan to play exactly 12,600 hands? Or might he have made a similar complaint after 11,800 hands or 13,100? The last describes what people actually do, so your figure of 3 in 1000 is extremely misleading.

You were off by more than an order of magnitude again. Is that a habit of yours? Please take the time to avoid gross mistakes like that.

[ QUOTE ]
For instance, if your down 74BB after 400 hands, its 99.7% certain your not a so-called "solid winner" and making less than 2BB per hand.

[/ QUOTE ]
First, a win rate of 2 BB/100 would not be within the 99.4% confidence interval, but that is very different from the statement that you are 99.4% certain that the player is not winning 2 BB/hand.

Second, losing 74 BB in 400 hands is extremely unlikely whether you win 2 BB/100 or lose 2 BB/100. It's not a useful test. This type of test does not allow you to classify most players by their results in the first few hundred hands.

Third, losing 74 BB in 400 hands is primarily evidence that you were really unlucky or were throwing a party for the table. It's not very strong evidence about whether your win rate is -2 BB/100 rather than -2 BB/100. Look at the Bayesian updates. (Rather, look this idea up on google, then evaluate the Gaussian density at these points.) <font color="white">Increase -2 relative to 2 by a factor of 3.7.</font>

[ QUOTE ]

It may be that each of those players has a low variance with respect to his particular games, but mix them together, and ta'da high variance.

[/ QUOTE ]
Your speculation is wrong again, as you could have checked.

When you assume the players have win rates of exactly 2 BB/100, and a standard deviation of 15 BB/100, then the result is a wide spread of possible observed win rates after 10,000 hands, including many players who are behind.

We didn't start with a wide spread of observed win rates for players we hoped were solid winners, assume that this is explained by variance alone, and then conclude that the variance must be erroneously high. It's not hard to estimate the standard deviation for an individual player, and these estimates are the source of the 15 BB/100 figure.

[ QUOTE ]
Thus, even though I'm using the 15BB/100 hand variance suggested by the previous poster, its not going to be the best method.

[/ QUOTE ]
You are confused. You have given no evidence other than wishful thinking to say that the standard deviation is lower than the 15 BB/100 computed by PokerTracker.

It's only a mild surprise if a solid winner is behind after 10k+ hands. If you don't understand this, then you don't understand variance.