Confidence Limits / Intervals & Betting

[rjp]

* Note: I posted this in Sports Betting, but I realize it's better here.

For those of you with a more formal foundation in statistical analysis, what do you consider to be an acceptable confidence level for approximating true odds?

For the sake of discussion, assume I observe 250 successes out of 500 trials. I can construct the following confidence intervals: 95% (45.53%,54.47%); 99% (44.16%,55.84%); 99.9% (42.59%,57.41%); 99.99% (41.27%,58.73%).

This has recently become a topic of major interest for me as I do not wish to overestimate my edge when betting half Kelly. As such, I've been using the lower tail (actually just one sided) of a 99.99% confidence interval as the true probability based on an observed sample.

I guess what I really want to know is: is this the best method for estimating a probability based on a sample for betting purposes, and if so, what's a good confidence level to use? Is 99.99% too high?

[rufus]

I don't have any formal statistics training. Regardless what you're asking doesn't really make a whole lot of sense to me.

In sports betting, the typical scenario is that we have some process where the odds are unknown. The bookie gives you one set of odds, and you make probability predictions, and then bet when you think you have an edge.

So, when you're betting, unless it's some kind of arbitrage, it's hard for me to understand how you can distinguish between a successful prediction of the odds, and a failed one in any absolute sense. You never know what the probability of a particular game result is. (Standard disclaimer about bayesian and frequetist notions applies.)

If you're referring to past observations of games, I have no experience with sports betting per se, but it seems very strange to me that you'd have any scenario in sports where you actually get repeatable anything, so you've got to make simplifying assumptions. How much these assumptions affect your prediction confidence is a big issue that can't really be addressed without looking at the assumptions. So I strongly suggest you stay very conservative vis-a-vis Kelly betting, and stay aware that bad assumptions can easily lead to lossy betting.

With good-sized data sets and computers, you could apply some sort of numeric approach to testing your model.

[rjp]

At a basic level I'm just looking for the best technique for calculating probabilities of an event based on an observed sample.

Assume I'm flipping a coin or doing some other random event. What I want to better understand is what probability can I safely assign an event based on a sample of observations?

Obviously this can't be an absolute number, so the confidence interval seems to fit what I'm looking for. The question then becomes what's a good level of confidence to use for going forward in the future, and what point on the interval should I be using as the estimate? (Call it worst case scenario if you will.)

[rufus]

Without extra sophistication, the confidence interval stuff is going to be your best bet. It does assume that trials are independent.

A conservative assumption would be to say that everything outside your confidence interval is a loss, i.e. work out your edge by multiplying the gross expected winnings by your confidence.

So, normally you'd be calculating something like:
EV=probability of winning * money won - money wagered
because of the confidence interval you add an extra factor:
EV=confidence of predition * probability of winning * money won - money wagered

Because the bets will have non-zero value even if your prediction is incorrect this gives an EV that's a bit low. However, it's non-trivial to correct.

Anyhow, you can then plug this modified EV into your Kelly calculation or whatever you like.

[neverforgetlol]

All a 99.9% CI really is is the range, it's not very useful.

[rjp]

neverforgetlol, I understand that, thus the reason of using the lower end to keep from overestimating. [img]/images/graemlins/smile.gif[/img]