Dr. Bob -- sharpshooter fallacy?

[Rococo]

I am pretty dubious about Dr. Bob's "technical" analysis. I haven't looked at it closely, but I suspect that I would find a sharpshooter fallacy problem if I did. Has anyone looked at this issue?

[SunOfBeach]

I think it's virtually a given that the're some sort of data mining/snooping bias in his models. There almost has to be at least a subtle mining bias, assuming that he began by considering things which he already had heard/read may be usable.

But, to his defense, he seems to understand multicollinearity issues and whatnot, using factors to mitigate the problem (such as looking at rushing and ToP together). So perhaps he's taken some time to avoid the mining bias as much as possible.

Also, his track record has been pretty darned sweet, over a reasonably large sample... so that gives at least some evidence to support his models being robust.

[MicroBob]

If I'm understanding this correctly:
His models being robust overall doesn't mean that some of this stuff might not be occuring with some of his observations though.

Some of the stuff seems to be reaching a bit such as:

"how home favorites perform after winning straight up as a double road underdog."

I assume he means road underdog of 10 points or more.
I think that looking at such a stat in a vacuum without context can be dangerous although I agree that if one wants to try to use the argument that a team is likely to have a letdown after a big upset win the numbers evidently indicate that may be unfounded.
And I guess if the betting public unjustly believes in a let-down scenario then there is value in the line.

So such a stat could be justified but man would I look at this cautiously and try to find other reasons like scheduling or whatever to explain why some of that stuff happened.

Obviously some of this stuff is just going to come down to variance as well and I'm sure Dr. Bob understands that but some of the numbers he throws out make me wonder.

I was with a baseball team one season that had gone 2-10 on Tuesdays but was 11-1 on Wednesdays and my hunch is that it had nothing to do with anything like the whole team partying on Monday nights or that they would 'bounce-back' after their loss or anything like that. It was just a weird variance quirk.

But it feels like some of the trend-freaks of the sports-betting world might have detected the pattern and started to fade that team on Tuesdays and bet the house on them on Wednesdays.

[Rococo]

[ QUOTE ]
I think it's virtually a given that the're some sort of data mining/snooping bias in his models.

[/ QUOTE ]

I think that you are being charitable. I suspect that it is 95% datamining. Now sometimes the data collects around the bullseye, but not always, especially when the sample size is small, and the cutoffs are arbitrary.

I particularly love it when he says something like, "there is a strong angle for Week 3 bounceback home favorites laying 3 pts or less, but be careful if the line moves to 4, because the angle for bounceback home favorites laying 4 points or less is not nearly as strong."

[SunOfBeach]

I wonder how much of this "Team X covered at home in 12 of their last 14 following road losses of 7+" is marketing lingo for his squareish clients, who may not understand some of the more technical aspects of his model. If he's using some sort of regression model, I don't see why he'd reveal what's really driving his picks. It's reasonable that this just could be marketing blahblah, to give him something to say other than "my model, the contents of which I can't disclose, says team x should be favored by 6 while the line is only 2.5..." yada yada yada.

[NajdorfDefense]

[ QUOTE ]
I wonder how much of this "Team X covered at home in 12 of their last 14 following road losses of 7+" is marketing lingo for his squareish clients, who may not understand some of the more technical aspects of his model. If he's using some sort of regression model, I don't see why he'd reveal what's really driving his picks.

[/ QUOTE ]

I agree 100%. I don't think he's datamining, I think he re-does that stats and puts them in his model. I just don't think he'll continue to run >58% forever.

One concern would be, if something in the game has changed since he started doing these picks - rules changes, scholarship changes, clock rule modifications -- and he has not adjusted his model for factors that affect the underlying stats and game.

[MyTurn2Raise]

IMO, you're on the right track Najdorf. I think scholarship changes are really making a difference. I'm used to hitting a pretty good clip myself as it seems it was easier to find a matchup disadvantage and go with it. Now, there are not the glaring differences and games are coming down to, more often than not, who wins the turnover and big play battle. It's who can create the big turnover. It's who has the guys that can break a punt return or turn a screen pass into 60 yards. Very few teams are the traditional line them up and smack the other team around (Ohio St comes to mind as one who does do it the old way). Lots of my models are based on stuff similar to Dr Bob. I look at ypp a lot. Anymore, I'm not sure it has as much statistical impact as it once did.

anyway, rambling response...but I think the more even teams have created differences in the game where it mainly comes down to play makers.

[SunOfBeach]

Just read these Dr Bob gems in another thread:

[ QUOTE ]
I looked up how home favorites perform after winning straight up as a double road underdog. As it turns out teams in that situation are good bets and Auburn applies to a 36-6 ATS subset of a situation based on the premise.

[/ QUOTE ]

[ QUOTE ]
Vanderbilt, meanwhile, is playing their first road game after 4 home games and the Commodores apply to a negative 23-57-3 ATS situation that plays against road teams after playing 4 or more consecutive home games.

[/ QUOTE ]

[ QUOTE ]
The only negative is Vanderbilt’s 11-2 ATS mark as a road underdog the previous 3 seasons

[/ QUOTE ]

Ouch...

[NajdorfDefense]

[ QUOTE ]
The only negative is Vanderbilt’s 11-2 ATS mark as a road underdog the previous 3 seasons

[/ QUOTE ]

Ouch...

[/ QUOTE ]

I take that to mean that otherwise Vandy would be getting more points in Vegas in his eyes.

[SunOfBeach]

Yup... missed that.