Normal Distribution -- Applicable to Winrates?

[Collin Moshman]

Hi guys,

It seems that in discussions of poker winrate calculations, it's usually taken for granted that the normal distribution applies. As a typical example, here is an article from this month's 2+2 magazine on SNG ROI calculations:

http://www.twoplustwo.com/magazine/c.../Vork0607.html

Keeping the discussion a bit more qualitative, I don't think this arrows around a bullseye approach works well for poker. As time goes on, your abilities change, as do your opposition, available rake deals, etc. E.g., when I play SNGs, I like to think I'm getting better with time. In that case, basing winrates on a mean parameter would clearly give misleading results. Same with BB/100, other games ... has anyone seen an approach to winrates that relies on other parameters / appeals to other distributions?

Any ideas greatly appreciated.

Best Regards,
Collin

[filsteal]

Collin,

You're right in that increasing skill or varying game environments can, over time, change your "true" winrate. I'm just not sure how useful it would be to try to take that into account.

Typical winrate analysis assumes that you have some "true" winrate, and that we can estimate your "true" winrate by saying it's approximately equal to your observed winrate. The larger the sample we observe, the more confident we are that the observed winrate is a good estimate of the "true" winrate.

There's nothing to prevent someone from taking this analysis up a notch. You could instead propose that your "true" winrate is NOT constant, but that it instead varies with time. You would need to decide HOW it varies with time (linearly? probably not) and define a model for your winrate as a function of time, up to some number of unknown parameters. You could then use regression techniques to estimate those parameters and to find the errors associated with those estimates. And, voila! You have a time-dependent winrate model.

However, I see three reasons why you wouldn't actually want to do this:

1. It's not clear to me what the best way would be to model the change in winrate with respect to time. And if the model's not at least approximately right, the results of the regression analysis won't be useful.

2. It's hard enough to estimate true winrates when we assume them to be constant -- we need boatloads of data to do it accurately. When we assume them to be complicated functions of time, we increase the number of parameters we're having to estimate, which means we'll need even more boatloads of data to get accurate estimates.

3. Any improvement we might get in terms of trying to estimate players' current or future "true" winrates will probably end up being pretty small compared to the variance associated with poker. (I'm sort of guessing on that, but it's a confident guess.) Here's an example of what I mean. With our complicated model, we might end up predicting that Player X will make $2600 this month, plus or minus $1400, instead of using a simpler model to predict that he will make $2500, plus or minus $1400.

Edit:

I just realized I didn't really answer your question.

The problem isn't with the normal distribution; it's with the modeling of the winrate as a constant thing. Most anytime that you have something that's an average of a bunch of things, it'll be approximately normally distributed (Central Limit Theorem).

[JavaNut]

I think the main point in any winrate discussion is what is your purpuse of trying to estimate a 'true winrate'.

The only purpose I can see is to be able to decide if you are capable of living of playing poker.

To be able to live a decent life now and to save up to retirement, you need a minimum average monthly income.

For that you need to be able to do 2 things, firstly to get an estimate and more important a confidence interval around this estimate of your monthly winnings and secondly to be able to determine whether you still are doing as well as you'd like.

You can fairly easy calculate how many hands you can play in an average month, remembering to add vacation time etc.

For each session you can calculate your winrate and for all sessions you can calculate the mean value and std. dev.

Doing this you can get a confidence interval by +/- 2 times the std. dev. And if that is above your minimum monthly average income, then Bob's your uncle.

At a later period you can redo this and see if Bob's still your uncle.

Another thing is that you can plot each sessions winrate along a time axis and do a liniear regression on that to detect if there is a trend, up or down. It will get you very depressed during a downswing but once you have got through several downswings that effect will be reduced.

But you still need a very large timescale, several years, to be able to say from such analysis that you are indeed a winning player, and that you can make a very good living from poker.

And the day after you finally are 100% confident that you are a winning player that can look forward to flashing your cash in the face of the other inmates at the retirement home in Florida, that are on a tight budget, someone publishes a book on poker that revolutionises the game [img]/images/graemlins/grin.gif[/img]

[Red_Diamond]

Well, sort of related to topic here, mention of the TRUE rates brings back memories of PT discussions. IIRC standard deviation while being accurate as reported, also wasn't right on the money. Reason being was PT used a clumping system. Thus, it would split up all hands and spread them out into many FICTITIOUS sessions of 100 hands each. While given an infinite amount of data, this will converted onto the TRUE rates, we all know how many samples one needs in poker that this would take.....forever.

As I recall, some players ran the numbers and there was quite some noticable difference on the confidence intervals between the 100 hand session method, and TRUE system after 30K+ hands still. Which, is sort of expected?

[filsteal]

[ QUOTE ]
Well, sort of related to topic here, mention of the TRUE rates brings back memories of PT discussions. IIRC standard deviation while being accurate as reported, also wasn't right on the money. Reason being was PT used a clumping system. Thus, it would split up all hands and spread them out into many FICTITIOUS sessions of 100 hands each. While given an infinite amount of data, this will converted onto the TRUE rates, we all know how many samples one needs in poker that this would take.....forever.

As I recall, some players ran the numbers and there was quite some noticable difference on the confidence intervals between the 100 hand session method, and TRUE system after 30K+ hands still. Which, is sort of expected?

[/ QUOTE ]

Very interesting. If PT does it this way, it's doing it both the stupid way and the hard way. It ought to just treat each hand as an observation and go from there.

[JavaNut]

A few more thoughts on normal distribution.

Firstly the results cannot be true normal distribution, as there is a limit on how much you can win/lose in one hand. Playing limit hold'em, you can not lose more than 1,200 BB/100 hands and can only win max 10,800 BB/100 hands.

But what you can also do is to test how well your session winrates fits to a normal distribution with mean value and std. dev. That would increase your confidence in the confidense interval you have established (in my earlier post). You might be able to find a better distribution which would give you a better model.

What you cannot do is to assume that you have a given distribution without testing how likely it is to be that kind of distribution. Though in cases like this a normal distribution is rather likely.

But if you get into a situation where you have few bigger losses and many small wins, perhaps you then aren't staying at the tables when winning as you should be.

[filsteal]

[ QUOTE ]
For that you need to be able to do 2 things, firstly to get an estimate and more important a confidence interval around this estimate of your monthly winnings and secondly to be able to determine whether you still are doing as well as you'd like.


[/ QUOTE ]

I think what Collin's suggesting is that the assumptions used to construct these estimates and confidence intervals are invalid.

I agree that he's right, at least in principle, but I don't think it's practical to do these sorts of analyses any differently.

[Red_Diamond]

[ QUOTE ]
Firstly the results cannot be true normal distribution, as there is a limit on how much you can win/lose in one hand. Playing limit hold'em, you can not lose more than 1,200 BB/100 hands and can only win max 10,800 BB/100 hands.

[/ QUOTE ]

LOL so true.

[JavaNut]

Look what you have done, now I've gone and opened my statistics book. [img]/images/graemlins/mad.gif[/img] (I've heard about one guy whose little daugther told everyone that her daddy was learning sadistics) [img]/images/graemlins/blush.gif[/img]

Modelling tourneys, either sng or mtt is not that easy as the result clearly isn't a normal distribution.

If you don't get in the money you will have -1 buy-in, or in the case of rebuy/add-on multiple buy-ins (except for the fact that you don't pay the fee for the rebuys/add-ons).
And then there is freerolls.

If you get into the money you get ~2..n buy-ins approximately. (But what is the buy-in for free-rolls).

You'll have to stick with +/- $ on each tournament, but you need some other distribution to model it.

Playing only sngs with a fixed price structure you could model this as a multinominal distribution (I hope it is called that in english, statistics was one of the only courses where we didn't have books in english)

Assuming a 50-30-20 price structure and 10 entrants and a fee of 10% of the buy-in you get either -1 buy-in (including fee), +1.8, +2.7 or +4.5 buy-ins.

You can then estimate the probability of each category based on your previous history.

Your winrate is then easily calculated based on the probability of each category.

You can then test a later sample towards that result to see if you are still around that or if your score has shifted.

[filsteal]

Javanut,

It doesn't matter whether we're talking about cash games, STTs, MTTs, or anything else. With a large enough sample size from ANY population with finite variance (here the "population" is all possible hands or tourneys), the average will have a limiting normal distribution, by the Central Limit Theorem.