New Thread On Sklansky Extrapolation Question

[ZeeJustin]

[ QUOTE ]
the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.


[/ QUOTE ]

I feel the same way, hence my OP about the atheism intelligence correlation being the #1 reason to believe.

[hitch1978]

I agree with model.

[Borodog]

David,

There is no way to say without knowing what Y is and the backgrounds of the people answering the survey.

Edit: On second thought, I agree with not a model.

Re-edit: On Third thought, maybe I don't.

Re-re-edit: Yes, yes I do. But as soon as you know ANYTHING about Y and the survey group, then the trend can be thrown out the window if necessary.

[Taraz]

[ QUOTE ]
David,

There is no way to say without knowing what Y is and the backgrounds of the people answering the survey.

[/ QUOTE ]

This is what I was trying to get at in the other thread. It depends on what kind of belief Y is.

I don't see why people think that a linear relationship between intelligence and believing in Y is indicative of Y's truth when the overall percentage of believers is still so low. It seems much more plausible that there would be a systematic bias of some sort unless Y is a very technical concept which requires a lot of background knowledge.

[PairTheBoard]

[ QUOTE ]
it seems to me like you are just being captious and overcomplicating matters to try to find a specific instance where the answer is "no" to david's question

[/ QUOTE ]

Actually that's Tom Crowley's job to accuse me of being captious. Ironically, I was merely reiterating the analysis Tom Crowley gave on the other thread for this topic. The irony goes even deeper when you consider that David Sklansky has crowned Tom Crowley King of SMP posters, declaring that Tom Crowley has never made an incorrect post.

It's really not being captious here anyway. Sklanksy himself is not sure if his notion for extrapolating the trend is valid. I also doubt if Sklansky would object to introducing the idea of evidence evaluators estimating a Price on which of the Yes-No answers is correct as a means of deciding which answer to give. In fact his new formulation of the problem in terms of people skilled at getting Yes-No questions correct lends itself very naturally to those people estimating a Price on the answers. Especially since we know nothing about the type of question being asked.

So introducing the concept of Price and a distribution of opinion for that price as the model for the situation is something that really needs to be done in order to argue coherently about the possibilities of what's happening. Once we do that we can see that what you might think must be happening isn't necessarily the case and cannot be the only factor involved in deciding if it's logical to deduce the extrapolation result Sklansky hypothesises. It's not the slam dunk it might be if there were only one logical possible factor possible under Sklansky's assumptions. You can argue further based on guesses for relative sizes of categories of propositions that satisfy one factor or the other. But that argument is much more speculative.

Furthermore, it's not clear that Tom Crowley's model fits all types of propositions. The notion that the distribution for the opinions of Price behaves in some kind of aproximately normal way may not be the case for all types of propositions. There may very well be propositions where double humps can develop in the distribution, which is what Sklanksy envisioned in his original formulation of the problem. Crowley claims this phenomenon is not realistic and implies an irrational model. I'm not so sure about that myself. There may be questions that produce polarity of opinion among even the best human minds but which can still be answered definitively in the limit toward infinite intellegence. If so that would bring in even more categories of propositions for you to guess the relative sizes of.

PairTheBoard

[Piers]

Nature is full of semi linear trends. Trends that are linear for a significant part of their range, but non linear outside that range. This is because curves often look linear over a small portion of their range, ever seen the approximation x~sin(x) for small x.

So when I see a naturally occurring scatter graph that a linear regression will closely fit, my initial assumption is that we are looking at a curve that has a semi linear part rather than a strictly linear relationship. The tendency to assume linear relationships reflects our tendency to simplify complex patterns so they can fit into our limited ability to understand rather than an underlying feature of process being analysed.

[David Sklansky]

[ QUOTE ]
Nature is full of semi linear trends. Trends that are linear for a significant part of their range, but non linear outside that range. This is because curves often look linear over a small portion of their range, ever seen the approximation x~sin(x) for small x.

So when I see a naturally occurring scatter graph that a linear regression will closely fit, my initial assumption is that we are looking at a curve that has a semi linear part rather than a strictly linear relationship. The tendency to assume linear relationships reflects our tendency to simplify complex patterns so they can fit into our limited ability to understand rather than an underlying feature of process being analysed.

[/ QUOTE ]

That's all well and good but it misses the point of this particular question. Even if it was the case that almost everything fits a linear relationship, it is very possible this does not. There is a paradoxical aspect to this problem.

Everybody knows the paradox of the statement "everything I say is a lie". But what of the statemnt "90% of the things I say are a lie"? If that is to be believed and it applies to the statement itself, then what are the odds the speaker has red underwear if he says he does? This isn't of only theoretical concern either. Tom Cowley says that 95% of Pair The Board's comments are moronic. Pair The Board goes on to laud Tom's intelligence. So you can see that these kind of things can give even me headaches.

[Piers]

[ QUOTE ]
Tom Cowley says that 95% of Pair The Board's comments are moronic. Pair The Board goes on to laud Tom's intelligence. So you can see that these kind of things can give even me headaches.


[/ QUOTE ]

[img]/images/graemlins/grin.gif[/img]

[PairTheBoard]

[ QUOTE ]
i don't understand why in this thread or the other thread people aren't just admitting that it is obviously very reasonable to think that Y is probably true.

[/ QUOTE ]

[ QUOTE ]
it seems to me like you are just being captious and overcomplicating matters to try to find a specific instance where the answer is "no" to david's question

[/ QUOTE ]

Looking closer at Tom Crowley's model I see that it gives us much more than just a "specific instance" where Sklansky's hypothesis fails. Crowley assumes that there is an aproximately normal type distribution for the opinions of the Price on this proposition being False within each group. He futher assumes that the mean of the distributions moves closer to the correct Price as you move from group G up to group A. He futher assumes that the distributions tighten up with shrinking standard deviations as you move from group G to A and that the distributions would converge to a point mass on the correct Price if you could go to the limit toward infinite intellegence.

Crowley believes this model covers all rational propositions, which I think is debatable. However it surely covers an extremely large category of rational propositions, maybe even in some common sense way nearly all of them. Common sense tells me that relatively few rational propositions produce double bump polarities of opinions among human experts.

So focusing on Crowley's common sense model and large category of propositions for which it fits, we ask how is Sklansky's scenario possible? There are two ways. Under Crowley's model, Sklansky's scenario logically implies that the G group is overestimating the Price on "False". The true Price on False may be <50% or >50%. In either case, as you move up the scale from group G to group A you would expect to see the lower tail of distributions for opinions to shift down into the <50% area. The question is, how fast does the distribution simultaneously tighten up? If it does so slowly we get Sklansky's scenario even when the true price for False is >50%. In the other case, we always get Sklanky's scenario when the true price is <50%.

Now think about it. How big is the category of propositions fitting Crowley's model for which G people answer No and where they are right? Out of that category, how likely is it that as we move from group G to A the mean of distributions for opinions on the Price of False moves more rapidly toward the correct price >50% than the Tightness of the distribution converges? What type of propositions might affect this relative speed of movement of mean and shrinkage of tightness for the distributions?

Maybe it's anybody's guess. But my guess is that it's a relatively large subcategory. Why? Because when there is unanimity among the G group as in Sklanky's scenario they can easily be Extreme in their overestimation of the Price for False, even when the evidence for the Price is difficult to evalutate. For such cases I would expect experts to quickly dismiss the extremist bias of lower groups, moving the mean of their distribution relatively quickly closer to the true Price. If the evidence is difficult to evalute the tightness of their distributions may not shrink so quickly. This defines a very large category of propositions satisfying Sklansky's scenario but for which his extrapolation hypothesis fails.

Remember, the subcategory above comes out of what should be the common sense much larger of the two categories. Certainly the vast majority of propositions for which G group people answer No with 100% unanimity are actually False. In fact, this is part of the reason why they can have exteme opinions overestimating the true price for False based on the evidence.

So Crowley's model does much more than just offer one counter example. It indicates what I think is a good case for the subcategory of propositions which satisfy his model, satisfy Sklansky's scenario, and are counter to Sklansky's hypothesis - and what you think is "obvious" - is indeed the common sense largest category of propositions under consideration. Knowing nothing else about the proposition, the better conclusion under Sklansky's assumptions, if you insist on making one, is that the propostion is "probably" False, just like everybody in the G group thinks.

PairTheBoard

[TomCowley]

I didn't think that normal-ish was the only possible distribution. Double-humps are plausible if there is one iffy piece of evidence that has two plausible evaluations, one of which ends up with a price of "about 70%", and the other "about 40%". Everybody is pretty confident that one of these two answers is correct, but is only slightly confident in their own answer- my specific objection was to large double-humped groups with high confidence.

Once you postulate wacky distributions, you're basically assuming the extrapolation isn't valid, which isn't necessarily wrong, but it's not an interesting result. I wanted to start with the friendliest assumptions to see if the extrapolation worked in those circumstances, so I could see what conditions were necessary to make it true, but it didn't work even then.

That being said, if the trend is 5% up to 30% with increasing IQ (yes/no skill), then I think it's safe to rate Y a higher probability to be true than if the trend is 30% down to 5%. That's also justifiable by simply giving smarter people more weight and not extrapolating. I'm not sure about the 5%-30% trend case compared to other cases that end up at 30% (flat 30%, decreasing to 30%, etc). As much "intuitive" sense as it makes to extend the line, if I want to set a price without knowing anything specific about Y, I have yet to come up with any justification for putting the probability of Y being true far away from 30% in any of those cases.

I'm also not convinced that it's even meaningful to discuss a price here, since this has a lot of similarity to the bent coin/personal probability stuff (that I really don't want to rehash), but if I had to set a price to prop-bet somebody with the same information, I haven't yet found a justification for it being far away from 30% (possibly a little different based on some form of weighted averaging across the groups, but not based on extrapolation).