New Thread On Sklansky Extrapolation Question

[David Sklansky]

There were too many side issues discussed on the other thread. Here is a clearer restatement of the question.

Say people are carefully evaluated and are classified by how likely they are to get things right on yes or no questions. They are rated from A to G. A's are historically the most likely to get things right. But even they are far from perfect.

A specific question comes up- "Is Y true?"

The general consensus is that it is not. 70% of A's think it is not.

The thing is that 100% of G's think Y isn't true. Same with 95% of F's, 90% of E's, 85% of D's, 80% of C's and 75% of B's.

There is a clear pattern. And it is heading toward a conclusion that if there were people substantially better than A's at getting answers (call them Omegas), most of them would in fact believe that Y IS true.

The question is whether it is reasonable, given no other information, to think the pattern will continue and that it would be a good bet to put your money on Y's truth. Or should we assume the pattern probably WON'T continue. Meaning that most omegas, if they existed, would agree with the majority of the A's (and everyone else).

Here is another way of looking at it. Without knowledge of the survey, seventy percent of A's think not only that Y is true but also that most omegas also think that y is true.
Should they change their mind when apprised of the survey and the pattern it seems to show? Remember that the survey will tell them that 70% of the smartest people agree with them.

In order to be persuaded to change their minds an A who is one of the 70% to believe Y, must somehow think changing is right, though 70% of the smartest agree with him, soley because even MORE than 70% of the less smart agree with him. Could that make sense? Yet if he sticks to his guns, he is defying a pattern when there is no good reason to think it wouldn't continue. (To make the impact clearer, one might imagine that the known pattern ends with 55% rather than 70% of the smartest people disagreeing with Y. Now only a small extrapolation has smarter people yet, agreeing with Y.)

[PairTheBoard]

Did you read Tom Crowley's analysis on the original thread? He points out that the fattening of the Yes percent may be due to the G group overestimating the correct Price say of 67% for a No answer. As you go up the scale from G to A, if the distribution doesn't tighten much, you will see the tail of the distribution move down into the Yes area while the mean moves accurately downward and closer to the correct 67% Price for No.

If better evidence evaluators were available you would eventually see the spread of the distribution tighten up, and the tail in the Yes area disappear as opinion converged on a 67% Price for No. The percent of people actually answering No would then concurrently rise back up to 100%.

So your extrapolation idea is simply unsound even under the kind of assumptions you want to make. I suppose you might speculate on the relative sizes of classes of propositions for which G's overestimate and underestimate the correct price, but that seems pretty futile to me.

PairTheBoard

[ZeeJustin]

2 thoughts come to mind.

1) The answer to this question doesn't necessarily have to be the same if Y is changed to "There is no God". I hope that is obvious to anyone that responds.

2) I think Y will almost always be true. There will be rare exceptions. In those rare exceptions, 2 things are probable
1: Nearly 100% of Omega's are wrong, and the G's are right, but for the wrong reason.
2: There is some sort of curve. Perhaps Y seems obviously false, so the G's are right without thinking. The A's have some advanced concept that makes some of them disagree with the G's. The Omega's are smart enough to understand the flaw in the concept. Basically there is some fancy play syndrome going on.

[ZeeJustin]

I also want to add that I can't think of a single case in my entire adult life where I had been told that "most the experts think Y", and my response wasn't either of the following:

"Well I guess Y must be true."

"There's a good chance you are wrong about most the experts thinking Y is true, but if you are right, then Y is probably true."

I guess I need to exclude poker from this, because I think I have enough knowledge in several areas of poker theory to reasonably disagree with even the most qualified experts.

Edit: That last paragraph I wrote made me think of something. Assuming that the A's are the top echelon, it might be rational for an A to think Y is false even after seeing the survey results, because it is possible he is better qualified than anyone else. But it is extremely unlikely that a B shouldn't change his mind after seeing the results of the survey, because it is unlikely that he is more qualified than the A's that disagree with him. G's should absolutely change their mind once hearing of the survey.

I think the answer to David's question is obviously, "Y is probably true even though 70% of A's disagree", but I think it is a lot more interesting if you instead ask, "You are an A. You were 95-98% confident that Y is false. Should you change your opinion after hearing the results of the survey?" I think the answer is probably yes more often than it is no.

[not a model]

english isnt my first language, but i still speak it with a lot of beauty and grace. with this in mind, what do you mean by "Price," pairtheboard?

it seems to me like you are just being captious and overcomplicating matters to try to find a specific instance where the answer is "no" to david's question of whether or not it's reasonable to assume that Y is probably true.

i would say of course it's a reasonable assumption that Y is true. and i could think of many easy examples where it is the case. some conceivable examples of Y:

the monty hall problem (especially if you asked it back in the 1950s)

asking whether the word "toward" is defined as "afoot"

asking whether water can stay liquid at -10 degrees C at standard pressure

i could think of thousands more questions that would conceivably show a near linear relationship over the groups A-G where most people in A wouldn't answer them correctly. basically it would be most any difficult question where most people in A didn't get it right. (obviously very few would have a perfect linear relationship, but that's the case in the hypothetical OP, and there certainly isn't anything wrong with it happening to be a linear relationship.)

im sure i could come up with questions that would conceivably satisfy the conditions in the OP where Y was actually false, but these would be rare and anamolous questions, like some sort of trick questions where knowledge or IQ act against you in coming up with the right answer. it is a rare instance when being smarter than someone leads you to be incorrect against them.

i don't understand why in this thread or the other thread people aren't just admitting that it is obviously very reasonable to think that Y is probably true. sure there are instances you could think of where it would be false, but these would be rare in comparison to the instances of it being true.

generally, the vast majority of questions where the majority of the A group got it wrong (say it was a very difficult question or it required understanding at great depth in a particular field like probability or vocabulary) would have a similar breakdown of extremely few dumb people getting them right, few average people, more smart people, even more very smart people (even if the number of very smart people were less than a majority). however, very very few questions where the majority of the A group got it right would have that breakdown. so if you answer "no" to david's question, you would only be right in one of these very rare instances.

the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.

[bunny]

[ QUOTE ]
the answer to david's question is such a clear "yes" that im a bit bewildered by all the controversy and debate it's causing. if you want to say it's unrealistic or situation-specific, fine. but so far as the actual question is concerned, it's a resounding "yes" that Y is probably true for any given question.

[/ QUOTE ]
It seems strange to do a survey, discover that most super-geniuses think Y is false and then conclude that it's true because even more geniuses think it is false and an overwhelming number of dumb people agree.

I still dont think it's justified to claim it is probably true based on a meta-trend. I think it's more likely an odd property of Y itself which leads clever people astray.

[FortunaMaximus]

[ QUOTE ]
I think it's more likely an odd property of Y itself which leads clever people astray.

[/ QUOTE ]

Perhaps it's not an odd property per se, but that Y could be an abstract question. If it was a defined statement of fact with an obvious y/n answer, then you could discern the breakdown of the implied reverse logic of less intelligent people being more statistically correct.

Otherwise, I think there exists a phenomenon in where the smarter a class of people are, the more divisive they will be. Intellectual Darwinism, perhaps?

[madnak]

I think this is a judgment call. I'm almost positive there is no logical answer.

[calcbandit]

I think the real interesting question is given the above survey results, is the probability of A being true greater than 30%?

If the survey results were that all groups A-G homogeneously responded 30% yes and 70% no, the probability of Y being true would pretty much have to be set at 30%, barring any other information on the topic.

So, do we have a case here where Y is more likely to be true specifically because less people believe it than the above, even distribution example?

[ZeeJustin]

[ QUOTE ]
So, do we have a case here where Y is more likely to be true specifically because less people believe it than the above, even distribution example?

[/ QUOTE ]

The point isn't how many people believe it, but what kind of people believe it. The people that believe it are those most likely to be correct. If you were to assume the pattern would follow (which of course you cant assume 100%), you could also assume that if there were people out there that were always correct, 100% of them would believe Y to be true.