ICM Quantified - First Set of Results

[rvg72]

I believe that ICM is a great way to model $ev but I also did not believe that it was completely accurate compared to actual $ outcomes in some situations and I wanted to find these differences and see if there are practical applications for this.

So, I wrote software that analyzes ICM and real outcomes for every player in every hand of an SNG and stores the results in SQL database for processing. My first run through this incorporated results from about 1700 SNG's and my intention is to continue to increase the sample size. Any contributions are appreciated.

So far the results have been extremely interesting at least to me. In this first go around I have categorized stack sizes into 5 groupings:

Very Small = bottom 10% of samples
Small = 10th to 30th percentile
Average = 30th to 70th percentile
Large = 70th to 90th percentile
Very Large = top 10th percentile

So these grouping are done per level or per number of players remaining so obviously relate to different stack sizes for each level.

At EVERY level, a subtle but distinct thing happens between ICM $ and Real $. There is an S curve... I think this is pretty huge. At every level very low stacks are highly overvalued, low stacks are overvalues, average stacks are pretty close but vary by level, large stacks are highly undervalued and very large stacks are somewhat undervalued.

I then looked at it by number of players remaining (instead of by level) and found similar trends. For example, here are the results when there are 4 players remaining (note: all results here are based on 800 chip games. I do expect similar trends albeit with different nuances for 1000 chip games but have not looked at those results yet):


and here is the data:



Thses differences are not small... Obviously this will change the results of ICM $EV calculations used to determine optimal push / fold scenarios. Lower stacks will find that hands that were previously easy folds now become easy pushes and conversely larger stacks have even more value than ICM gives them credit for and therefor many of the marginal and not so marginal pushes need to becomes folds.

I plan on charting this for every blind level and # players remaining and then moving on to things like determining where you are in relation to the blind and blind size and how that affects real $ results.

Comments?

rvg

[citanul]

this [censored] rules.

keep it up.

c

[DonkBluffer]

[ QUOTE ]
this [censored] rules.

keep it up.

c

[/ QUOTE ]

I agree! I really feel like we (you) are achieving something here.

edit: maybe a stupid question, but could this be caused by calling ranges? A (very) short stack will often be called by at least one, and often two players if he goes all-in. A big stack on the other hand has more FE.

[rvg72]

[ QUOTE ]
[ QUOTE ]
this [censored] rules.

keep it up.

c

[/ QUOTE ]

I agree! I really feel like we (you) are achieving something here.

edit: maybe a stupid question, but could this be caused by calling ranges? A (very) short stack will often be called by at least one, and often two players if he goes all-in. A big stack on the other hand has more FE.

[/ QUOTE ]

Calling ranges / Fold Equity are definately key reasons for this but they are also reflected in your low ICM value - it is low because you don't have as much Fold Equity. What this shows is that your low stack is even worse than you thought.

Now this doesn't mean that you push any 2 from any position at any blind size if you are a low stack on the bubble. You are not going to steal any more often or win the all-in any more often with this new knowledge... but... when you are called and lose you don't lose as much since your $ev was lower anyways and if you succeed in the steal or double up then you make more real $ev than you would have expected.

rvg

[mgross]

rvg: Have you tried to model the S-curve? Does it fit a function that we can model as easily as ICM?

[mgross]

I wonder if it is something as simple as using ICM with a "difference from the average" multiplied in to compensate.

[rvg72]

[ QUOTE ]
rvg: Have you tried to model the S-curve? Does it fit a function that we can model as easily as ICM?

[/ QUOTE ]

Well this was my goal going into it - it would not replace ICM, I want to be clear about that. ICM is proving to be very, very accurate especially with medium stacks. What I would like to do is determine an adjustment that can be applied to the ICM calculation based on delta between stack size and average stack size.

More likely I will provide the data and someone else will be able to put this together. There are a lot of really smart people here.

rvg

[zabt]

Very cool. And, I suspect not a big surprise. I'm very interested in seeing the by position results.

[rvg72]

[ QUOTE ]
Very cool. And, I suspect not a big surprise. I'm very interested in seeing the by position results.

[/ QUOTE ]

The "By Position" will be very interesting... I suspected that big stacks were undervalued when blinds were high and small stacks were overvalued when blinds were high which was true. What I completely did not expect was that this holds true even at Level 1... I had suspected the opposite trend when blinds were low but that was clearly not the case - in fact it seems that the difference is greater at earlier levels.

rvg

[zabt]

So, you mean that ICM more accurately values a small stack at higher blinds than at low blinds? Or, put another way, it is worse (relative to ICM) to have a short stack early than a short stack late? I suppose some of this is because when you have a short stack late, you're already late into the SNG, closer to the end, less random poker to be played, less time for error to accumulate so to speak. Or, maybe it's just because really bad players, the ones who are worse than random, are often short stacked early and their results are skewing the data. Come to think of it, the donks are more often out early so their datapoints will not be present as often in the higher blind levels. It might be that the good players are the ones that accumulate the large stacks and thus the reason that big stacks perform better than ICM predicts is that better players are playing the big stacks. And, the bad players are the ones that find themselves with short stacks which is why short stacks come up short of ICM prediction.

I guess you'll need to break down the data points by player ROI (or some other measure of skill) to see if short-stack worse than ICM/large stack better than ICM holds across player abilities.