New Thread On Sklansky Extrapolation Question

[David Sklansky]

There were too many side issues discussed on the other thread. Here is a clearer restatement of the question.

Say people are carefully evaluated and are classified by how likely they are to get things right on yes or no questions. They are rated from A to G. A's are historically the most likely to get things right. But even they are far from perfect.

A specific question comes up- "Is Y true?"

The general consensus is that it is not. 70% of A's think it is not.

The thing is that 100% of G's think Y isn't true. Same with 95% of F's, 90% of E's, 85% of D's, 80% of C's and 75% of B's.

There is a clear pattern. And it is heading toward a conclusion that if there were people substantially better than A's at getting answers (call them Omegas), most of them would in fact believe that Y IS true.

The question is whether it is reasonable, given no other information, to think the pattern will continue and that it would be a good bet to put your money on Y's truth. Or should we assume the pattern probably WON'T continue. Meaning that most omegas, if they existed, would agree with the majority of the A's (and everyone else).

Here is another way of looking at it. Without knowledge of the survey, seventy percent of A's think not only that Y is true but also that most omegas also think that y is true.
Should they change their mind when apprised of the survey and the pattern it seems to show? Remember that the survey will tell them that 70% of the smartest people agree with them.

In order to be persuaded to change their minds an A who is one of the 70% to believe Y, must somehow think changing is right, though 70% of the smartest agree with him, soley because even MORE than 70% of the less smart agree with him. Could that make sense? Yet if he sticks to his guns, he is defying a pattern when there is no good reason to think it wouldn't continue. (To make the impact clearer, one might imagine that the known pattern ends with 55% rather than 70% of the smartest people disagreeing with Y. Now only a small extrapolation has smarter people yet, agreeing with Y.)

[PairTheBoard]

Did you read Tom Crowley's analysis on the original thread? He points out that the fattening of the Yes percent may be due to the G group overestimating the correct Price say of 67% for a No answer. As you go up the scale from G to A, if the distribution doesn't tighten much, you will see the tail of the distribution move down into the Yes area while the mean moves accurately downward and closer to the correct 67% Price for No.

If better evidence evaluators were available you would eventually see the spread of the distribution tighten up, and the tail in the Yes area disappear as opinion converged on a 67% Price for No. The percent of people actually answering No would then concurrently rise back up to 100%.

So your extrapolation idea is simply unsound even under the kind of assumptions you want to make. I suppose you might speculate on the relative sizes of classes of propositions for which G's overestimate and underestimate the correct price, but that seems pretty futile to me.

PairTheBoard

[ZeeJustin]

2 thoughts come to mind.

1) The answer to this question doesn't necessarily have to be the same if Y is changed to "There is no God". I hope that is obvious to anyone that responds.

2) I think Y will almost always be true. There will be rare exceptions. In those rare exceptions, 2 things are probable
1: Nearly 100% of Omega's are wrong, and the G's are right, but for the wrong reason.
2: There is some sort of curve. Perhaps Y seems obviously false, so the G's are right without thinking. The A's have some advanced concept that makes some of them disagree with the G's. The Omega's are smart enough to understand the flaw in the concept. Basically there is some fancy play syndrome going on.

[ZeeJustin]

I also want to add that I can't think of a single case in my entire adult life where I had been told that "most the experts think Y", and my response wasn't either of the following:

"Well I guess Y must be true."

"There's a good chance you are wrong about most the experts thinking Y is true, but if you are right, then Y is probably true."

I guess I need to exclude poker from this, because I think I have enough knowledge in several areas of poker theory to reasonably disagree with even the most qualified experts.

Edit: That last paragraph I wrote made me think of something. Assuming that the A's are the top echelon, it might be rational for an A to think Y is false even after seeing the survey results, because it is possible he is better qualified than anyone else. But it is extremely unlikely that a B shouldn't change his mind after seeing the results of the survey, because it is unlikely that he is more qualified than the A's that disagree with him. G's should absolutely change their mind once hearing of the survey.

I think the answer to David's question is obviously, "Y is probably true even though 70% of A's disagree", but I think it is a lot more interesting if you instead ask, "You are an A. You were 95-98% confident that Y is false. Should you change your opinion after hearing the results of the survey?" I think the answer is probably yes more often than it is no.

[not a model]

english isnt my first language, but i still speak it with a lot of beauty and grace. with this in mind, what do you mean by "Price," pairtheboard?

it seems to me like you are just being captious and overcomplicating matters to try to find a specific instance where the answer is "no" to david's question of whether or not it's reasonable to assume that Y is probably true.

i would say of course it's a reasonable assumption that Y is true. and i could think of many easy examples where it is the case. some conceivable examples of Y:

the monty hall problem (especially if you asked it back in the 1950s)

asking whether the word "toward" is defined as "afoot"

asking whether water can stay liquid at -10 degrees C at standard pressure

i could think of thousands more questions that would conceivably show a near linear relationship over the groups A-G where most people in A wouldn't answer them correctly. basically it would be most any difficult question where most people in A didn't get it right. (obviously very few would have a perfect linear relationship, but that's the case in the hypothetical OP, and there certainly isn't anything wrong with it happening to be a linear relationship.)

im sure i could come up with questions that would conceivably satisfy the conditions in the OP where Y was actually false, but these would be rare and anamolous questions, like some sort of trick questions where knowledge or IQ act against you in coming up with the right answer. it is a rare instance when being smarter than someone leads you to be incorrect against them.

i don't understand why in this thread or the other thread people aren't just admitting that it is obviously very reasonable to think that Y is probably true. sure there are instances you could think of where it would be false, but these would be rare in comparison to the instances of it being true.

generally, the vast majority of questions where the majority of the A group got it wrong (say it was a very difficult question or it required understanding at great depth in a particular field like probability or vocabulary) would have a similar breakdown of extremely few dumb people getting them right, few average people, more smart people, even more very smart people (even if the number of very smart people were less than a majority). however, very very few questions where the majority of the A group got it right would have that breakdown. so if you answer "no" to david's question, you would only be right in one of these very rare instances.

the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.

[bunny]

[ QUOTE ]
the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.

[/ QUOTE ]
It seems strange to do a survey, discover that most super-geniuses think Y is false and then conclude that it's true because even more geniuses think it is false and an overwhelming number of dumb people agree.

I still dont think it's justified to claim it is probably true based on a meta-trend. I think it's more likely an odd property of Y itself which leads clever people astray.

[FortunaMaximus]

[ QUOTE ]
I think it's more likely an odd property of Y itself which leads clever people astray.

[/ QUOTE ]

Perhaps it's not an odd property per se, but that Y could be an abstract question. If it was a defined statement of fact with an obvious y/n answer, then you could discern the breakdown of the implied reverse logic of less intelligent people being more statistically correct.

Otherwise, I think there exists a phenomenon in where the smarter a class of people are, the more divisive they will be. Intellectual Darwinism, perhaps?

[ZeeJustin]

[ QUOTE ]
the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.


[/ QUOTE ]

I feel the same way, hence my OP about the atheism intelligence correlation being the #1 reason to believe.

[hitch1978]

I agree with model.

[PairTheBoard]

[ QUOTE ]
it seems to me like you are just being captious and overcomplicating matters to try to find a specific instance where the answer is "no" to david's question

[/ QUOTE ]

Actually that's Tom Crowley's job to accuse me of being captious. Ironically, I was merely reiterating the analysis Tom Crowley gave on the other thread for this topic. The irony goes even deeper when you consider that David Sklansky has crowned Tom Crowley King of SMP posters, declaring that Tom Crowley has never made an incorrect post.

It's really not being captious here anyway. Sklanksy himself is not sure if his notion for extrapolating the trend is valid. I also doubt if Sklansky would object to introducing the idea of evidence evaluators estimating a Price on which of the Yes-No answers is correct as a means of deciding which answer to give. In fact his new formulation of the problem in terms of people skilled at getting Yes-No questions correct lends itself very naturally to those people estimating a Price on the answers. Especially since we know nothing about the type of question being asked.

So introducing the concept of Price and a distribution of opinion for that price as the model for the situation is something that really needs to be done in order to argue coherently about the possibilities of what's happening. Once we do that we can see that what you might think must be happening isn't necessarily the case and cannot be the only factor involved in deciding if it's logical to deduce the extrapolation result Sklansky hypothesises. It's not the slam dunk it might be if there were only one logical possible factor possible under Sklansky's assumptions. You can argue further based on guesses for relative sizes of categories of propositions that satisfy one factor or the other. But that argument is much more speculative.

Furthermore, it's not clear that Tom Crowley's model fits all types of propositions. The notion that the distribution for the opinions of Price behaves in some kind of aproximately normal way may not be the case for all types of propositions. There may very well be propositions where double humps can develop in the distribution, which is what Sklanksy envisioned in his original formulation of the problem. Crowley claims this phenomenon is not realistic and implies an irrational model. I'm not so sure about that myself. There may be questions that produce polarity of opinion among even the best human minds but which can still be answered definitively in the limit toward infinite intellegence. If so that would bring in even more categories of propositions for you to guess the relative sizes of.

PairTheBoard