7k Post: Retooling how you think about sample sizes (tl;dr-too mathy)

[Aaron W.]

It's a few posts too early, but since I finished typing it up, I might as well just post it.

---------------------

We see the sample size warning go up over and over and over again. 3000 hands is not enough to determine with much confidence whether you are a winning player. Villain’s PFR is unreliable because you only have 20 hands on him. Some of these sample size warnings are true. But there are situations where you can use inadequate sample sizes combined with experience to get a “highly indicative” perspective of villain’s playing style.

Here is an example from 6-max: You’re playing your first hand at the table. Villain open-raises from the button and you fold J8o in the BB. The next hand, villain is in the cutoff and open-raises again and you fold your junk in the SB. Over the next orbit, villain doesn’t raise anymore, but overlimps once and folds a couple times. He had an opportunity to steal once during that second orbit and passed on it (what a tight game!). At the start of your third orbit, you’re in the big blind and villain open-raises from the button and SB folds. You have J8o. I would argue that you can call this all day long and I don’t think it’s very close.

(By the way, Stox considers J8o a standard defense against a “typical” steal raise. However, his notion of “typical” is not the same as the microlimit notion of “typical.” He recommends 30% from the cutoff and 40% from the button. The average microlimit player is probably much closer to 15% from the cutoff and 30% from the button (think about all the villains you see openlimp from those positions). Therefore, defending with J8o becomes less favorable because you are against a stronger hand range.)

This is the third time that you have seen villain open-raise from late position in a couple orbits. “But maybe he has had a hot run of cards.” Maybe... but let’s take a closer look at that possibility.

Let’s suppose that he’s a somewhat aggressive stealer. For reference, my ASB (attempt to steal blinds) is a little under 40% and we’ll use this number for now. We’ll even be generous and suppose that he has had 4 stealing opportunities and has only raised three times. What is the probability that a 40% stealer would have a stealing hand 3 out of 4 times?

This is a standard probability problem involving the binomial theorem. The probability of this happening is

(4 choose 3) * (40%)^3 * (60%) = 15%

So if villain is a 40% stealer, there’s only a 15% chance that we would see him steal 3 out of 4 times. What if villain is a 75% stealer? How often would he steal 3 out of 4 times?

(4 choose 3) * (75%)^3 * (25%) = 42%

Now unfortunately, we’re playing this game backwards. We don’t know villain’s ASB. Instead, we only know that he has attempted to steal 3 out of 4 times. Can we deduce anything about his ASB from this information alone?

Actually, we can’t. We need to make some assumptions on the distribution of ASBs over the entire range of villains. I don’t have a substantial database of players from which to draw decisive conclusions about ASBs, so I will make a simple model and see what happens.

I will assume that there are 3 classes of players out there:
1) 10% ASB (Raising premiums only)
2) 35% ASB (Solid stealer)
3) 80% ASB (Hyper-stealer)

I will further assume that the three classes come in a 2:2:1 ratio. That is,
1) 40% of the microlimit population doesn’t steal often enough
2) 40% of the microlimit population steals well
3) 20% of the microlimit population steals way too often

Under these assumptions, how do we guess villain’s ASB class? This is some standard statistics stuff. The formula is pretty simple:

Probability villain is in class A = (Probability a player in class A raises 3/4 times * % of players in class A) / (Probability that any player raises 3/4 times)

We now compute the denominator first:

4 * ((.1)^3 * (.9) * 40% + (.35)^3 * (.65) * 40% + (.8)^3 * (.2) * 20%)
= 4 * (0.00036 + 0.0111475 + 0.02048)
= 0.12795

We now compute the numerators:
1) 4 * (.1)^3 * .9 * 40% = 0.00144
2) 4 * (.35)^3 * (.65) * 40% = 0.04459
3) 4 * (.8)^3 * (.2) * 20% = 0.08192

Therefore, the probabilities are
1) 0.00144/0.12795 = 11%
2) 0.04459/0.12795 = 35%
3) 0.08192/0.12795 = 64%

After observing these only 4 hands (under our hypotheses), we know that 2/3 of the time, this villain is a hyper-stealer and we can begin to adjust immediately! As an exercise, do this calculation again (under the same hypotheses) for 0, 1, 2, and 4 raises out of 4 times. You might be surprised at the results.

Of course, there are some concerns here. My hypotheses could are bad. There are more than 3 classes of players. We also don’t know whether the distribution I gave actually resembles real microlimit poker. (My gut tells me that a better model would have 4 classes, 15/25/35/80, and that the ratio would be something like 2:4:3:2 – I haven’t run the numbers for this situation. I suspect the results are slightly less dramatic, but will still point to highest ASB class as the most probable.) These are things that you will have to investigate on your own, by observing players at the site/stakes/table and building up your experience with how they play.

Be careful not to take this idea too far. In this example, we were lucky. We were watching villain closely enough to see that he was attempted to steal 3 out of 4 times. In fact, we were lucky to see 4 opportunities in 3 orbits. If we’re at a table and villain is stealing for the third time in four chances on the tenth orbit, we would have more information than just his ASB to indicate his preflop aggression level. This information is valuable and will have to be balanced with the ASB calculation.

Last warning: I haven’t given you the slightest indication of how to play this hand postflop. Blind defense is very difficult and I don’t want to go into the nuances of it. Besides, Stox’s book has a lot of information on it, so why should I re-invent the wheel? (Plus, Stox is far better at it than I am!) The point of this post was to encourage you to think more carefully about “sample sizes” and how you determine whether a sample size is sufficiently large to draw some sort of conclusion.

Let’s now look at some situations. See what sort of analysis you can do with these situations. Remember that this is all microlimit 6-max. The point here is to try to understand how much you should shift your “standard” play based on some slight evidence. Maybe you shift it a lot, maybe it doesn’t shift at all because there isn't enough information. As always the key is to try to justify your answer using reasons, not just give answers.

1a) You are on the button with ATo and an unknown MP (second to act out of 6 players) raises. Do you 3-bet or fold?
1b) You’re in your 5th orbit on the button. The table has been playing somewhat tight. MP with stats of 20/10 raises and it’s folded to you. You have ATo. Do you 3-bet or fold?
1c) You are on the button with ATo. You’ve been multitabling and you haven’t really been noticing MP’s play in particular, but you have noticed the table has been generally tight and aggressive. He raises and it’s folded to you. Your HUD reads 25/20/3 after 60 hands. Do you 3-bet or fold?

2a) You are in the SB with A8s. Unknown button open-raises. Do you 3-bet or fold?
2b) You are in the small blind with A8s. The game has been quite LAGgy in 50 or so hands that you’ve been here. If anyone is paying attention, you also have a solid image because you’ve been taking down pots mostly uncontested, and the hands you have shown have all been quite strong. Button open-raises. He has been quite active, over-limping quite often and never open-limping. The only hand you remember is the one where he raised 77 UTG.
2c) You are in the SB with A8s. Everyone folds to the button who raises. You’ve been at the table for 4 orbits, and you can remember that he has open-limped in late position once and open-raised from the button once before, which went uncontested by you and the big blind. Do you 3-bet or fold?

3a) You open raise from MP and get 3-bet by an unknown SB. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?
3b) You open raise from MP and get 3-bet by the small blind. You’ve been at the table for about 30 hands, and have raised 5-6 of them. This is the first time he has 3-bet. You’ve seen this player coldcall raises (not necessarily yours) 3-4 times and he has done a lot of check-calling and check-folding. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?
3c) You open-raise from MP and get 3-bet by (you guessed it) SB. SB has been a pretty active and aggressive player in the three orbits you’ve been at the table. He has open-raised a couple times, iso-raised a limper once, and this is the second time you’ve seen him 3-bet. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?

By the way, I typed up most of this post before reading the magazine article which puts up some similar ideas regarding sample size in SnGs.

[NIX]

Excellent post Aaron. I use a similar thought process while playing but have never seen the math before to back it up. Was definitely worth reading and thinking about.

[Befolder]

Good post Aaron.

[bung]

Serious post bro ^^

[bravos1]

so how much did Stox give you to shill his book?

j/k.. ni han!

[OziBattler]

bumped for snitch pwned ftw. Aaron tyvm

[Guruman]

i like preflop, but then i'm a geek. [img]/images/graemlins/smile.gif[/img]

1a) You are on the button with ATo and an unknown MP (second to act out of 6 players) raises. Do you 3-bet or fold?

My default is to fold this until getting some semblance of a read. I do try to get reads quickly, but villain’s range is likely to skew somewhere near a dominating range vs my hand and if he has any heart or aggression in him at all I’ll end up folding the best hand some % of the rest of the time, so I just get out of the way until I see him playing fit or fold or raise something dumb pf.

1b) You’re in your 5th orbit on the button. The table has been playing somewhat tight. MP with stats of 20/10 raises and it’s folded to you. You have ATo. Do you 3-bet or fold?

Mp’s been playing tightly and he’s raising it up from an earlyish position. Even so, I could dominate some of his range here, and I can probably trust him postflop so I’ll go ahead and threeball and use position to test him here. A lot of this decision is based on a rough estimate of my ability to find a fold when he pushes back or starts digging his heels in postflop.

1c) You are on the button with ATo. You’ve been multitabling and you haven’t really been noticing MP’s play in particular, but you have noticed the table has been generally tight and aggressive. He raises and it’s folded to you. Your HUD reads 25/20/3 after 60 hands. Do you 3-bet or fold?

I start looking for a new table, but that’s because I’m trying to focus on my “playing good opponents” version of tilt right now, and I need the practice getting up. Given that I’ll just fold so that I can get up.

2a) You are in the SB with A8s. Unknown button open-raises. Do you 3-bet or fold?

This is an auto threebet from me. Typical button stealing ranges are very very wide, and my hand wants to play with initiative and headsup. To fold here we have to put villain on a range of something like A4+ and 44+, which is just far far too narrow to give an unknown.

2b) You are in the small blind with A8s. The game has been quite LAGgy in 50 or so hands that you’ve been here. If anyone is paying attention, you also have a solid image because you’ve been taking down pots mostly uncontested, and the hands you have shown have all been quite strong. Button open-raises. He has been quite active, over-limping quite often and never open-limping. The only hand you remember is the one where he raised 77 UTG.

easy threebet. button won’t have been paying attention, but this means that he’ll likely call down with any piece and I’m a fav against that strat postflop. I can also do thing like induce bluffs and screwplay postflop when I make hands.

2c) You are in the SB with A8s. Everyone folds to the button who raises. You’ve been at the table for 4 orbits, and you can remember that he has open-limped in late position once and open-raised from the button once before, which went uncontested by you and the big blind. Do you 3-bet or fold?

another super easy threebet from me. I can see that you’re looking to shrink button’s range by giving him a pf limping range, but I just don’t think that you can shrink it enough to pull the profitability away from A8. Also, players that limp pf tend to be far too fit-or-foldish postflop in general and will give up when they whiff even after a pfr. The other side is that they’ll tend to showdown any time they connect. Both of these factors add up to a myriad of ridiculous spews based on villain reading his own hands instead of ours, which is lots of incentive to get into the pot with him.

3a) You open raise from MP and get 3-bet by an unknown SB. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?

I’ll generally look for Ax, K9+, QJ+ and most pps out of sb threebets. I think the decision about what to do with the last pf action can be generally style dependant. For instance, in a HU match I’d either be calling here with 100% of my range or capping with 100%, depending on my read of villain’s flop and turn play (aggro flop and turn get calls, tight flop and turn get caps). Also, I think my mp raising range is going to be relatively comparable to sb’s default threebetting range - lots of broadway type hands and pairs. I’ll probably have a few suited connectors in there that he wont, but that’s near the extent of the difference. In the end, the decision comes down to how I’m playing againt the rest of the table, and how long I plan to continue at this table. If I’m getting up soon I’ll go super straightforward and cap 99+, AT+. If not I’ll think about what gear I’m in and go with that.

3b) You open raise from MP and get 3-bet by the small blind. You’ve been at the table for about 30 hands, and have raised 5-6 of them. This is the first time he has 3-bet. You’ve seen this player coldcall raises (not necessarily yours) 3-4 times and he has done a lot of check-calling and check-folding. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?

In my experience witnessing a coldcall narrows an opponents’s raising range much more than witnessing an open limp does. This is because lots of those coldcalls should be clear threebets, and not always clear folds. If villain has been coldcalling from the small blind, then he’s also displaying a lack of recognition of the presence of the big blind, which should further strengthen his range. This is doubly true at micro-stakes where players generally have less heart and more gambool. I’ll cap TT+ and AJ+ there, as I think he could have hands as strong as A8s or A9s in his coldcalling range, as well as some pocket pairs.

3c) You open-raise from MP and get 3-bet by (you guessed it) SB. SB has been a pretty active and aggressive player in the three orbits you’ve been at the table. He has open-raised a couple times, iso-raised a limper once, and this is the second time you’ve seen him 3-bet. What is the approximate hand range you will give him? What hands would you cap for value and which ones would you just call?

If I can recall all of this about sb, then I can also recall how he played those hands postflop. If he was getting thin valuebets in and finding ways to showdown his weaker hands then I’ll label at him as a good lag and start looking for a more comfortable chair. If he’s showing down hopeless hands or spewing bluffs in bad spots postflop then he’s a bad lag and I’ll start eyeballing that chair to his left. This is doubly true if it’s a fish that he’s been isolating pf that’s also sitting to my right.

I’d put his range at any Ace, most kings, maybe Q9+, any two broadways, any pocket pair, and depending on what I’d seen I could throw some suited connectors in there like T9s. I’ll generally call probably 90% of my range and look to raise him out of his seat on the flop and turn. He is the bull and I am the matador, and if I allow him to charge me it is so that I can slaugher him as he passes by.

[marchron]

Wow, most of this is over my head (not a big 6-max fan), but this . . .

[ QUOTE ]
He is the bull and I am the matador, and if I allow him to charge me it is so that I can slaugher him as he passes by.

[/ QUOTE ]
. . . is super-awesomes.

[00Snitch]

lololol totally missed this.

Reading...

[unterfish]

Nice post, Aaron, but I would like to add:
The PT/PAHUD sample-sizes might not be big, but they are just the best estimates in order to make your decision, b/c we dont have any further information - online. (The "filtration" of information is giving us no more input.)

However, to analyze a hand, theoretically, we can in a first step analyze the hand regarding only the first (and best) estimate of the PAHUD stats.
In a 2nd attempt, one could add some hands in a given "confidence interval", and giving them lower weight, then analyze the equity, and so on.

But, anyways, the PAHUD stats are just the best estimate. We cant know any better.

And regarding the model of Poker Players: that would be a nice project to start with - anyways Im already trolling I guess [img]/images/graemlins/laugh.gif[/img] [img]/images/graemlins/frown.gif[/img] .