Some Verification From Mathmeticians Please

[David Sklansky]

Pair The Board has been insisting that the way I approach some problems have no right to be called athoritatively mathematical because, while the arithmetic is right, the underlying assumptions are just my opinion. I maintain that it is only in the most nitpicking sense can one maintain that any other assumption is worth considering. Sort of like why solipsism shouldn't be considered. But he says mathmeticians are on his side. Because one happened to chime in regarding a very specific point.

This general argument moved into the specific realm of jurors opinions of a defendents guilt. I thought it would be a good idea to make them think about this opinion in probability terms. And the following exchange ensued. I'm hoping some mathmeticians and scientists out there will take my side. Even if there are complex technical reasons why my position isn't logically flawless.

Quote:
--------------------------------------------------------------------------------


Quote:
--------------------------------------------------------------------------------

I gave a precise definition of the pseudo probability as you call it. The juror is asked to imagine that there are 100 trials with the exact same evidence. How many of those defendents IN HIS OWN PERSONAL OPINION will be innocent. As it is, the juror is expected to give the answer "not many" before he convicts. So all I am saying is that it would be nice for jurors to be told what number should be considered not many.


--------------------------------------------------------------------------------



I cannot imagine how there could possibly be 100 different trials with the exact same evidence. That scenario is a figment of your imagination. When I imagine 100 copies of the 1 situation that is in front of me, all I can see are either 100 guilty defendents or 100 innocent ones. I just don't know which. That is a philosophical difference in how we look at it. I am not bound by your philosophical view nor by the imaginary figment you have conjured.

PairTheBoard


--------------------------------------------------------------------------------



You are simply wrong. The thought experiment while contrived, is not unimaginable. And it has nothing to do with philosophy. To show this, imagine that there is a horse race picking contest based totally on the information in the Daily Racing form. I'll say the races were already run to avoid a nitpick about past events vs future events.

There are one hundred DIFFERENT races with totally different horses. Eight horses in each race. But amazingly the past performances for horses one through eight are EXACTLY the same as far as what is in the racing form. In other words all number ones look alike. All number twos look alike. Etc etc. But they are NOT IDENTICAL Horses. And there are differences among them as well as among the conditions of the races. Some relevant. Such as height and weight. Or what the track bias was that day. But that information isn't available to you. Just like all evidence is not available at the trial.

Anyway you are now asked to pick the winner of each race. Say number three looks much the best. So you pick him. But that means you would pick number three in ALL races. Now your contention, if translated to this example, would be that number three will either win all races or lose all races. But this would obviously not be the case even if they were running on a straightaway with no racing luck involved.

If you change all the races to two horses races between Innocent and Guilty, I would hope that you would now understand that my thought experiment does not presuppose some debatable "philosophy".

[Shine]

Who cares about the "100 trials" argument. Is asking the juror his opinion of the probability of guilt really so hard? Would like to hear PTB here.

[Shine]

I take your side, DS, but I don't like the way that you've presented this.

As I read the above, what I think (could be wrong) you guys are saying is: DS thinks that in the hypothetical trials, unreported facts are independent and free to change, while PTB thinks that the man is either guilty 100/100 or not guilty 100/100 because he thinks that it is not reasonable to consider some things "independent", and that the world exists in its current state only.

[PairTheBoard]

[ QUOTE ]
Anyway you are now asked to pick the winner of each race. Say number three looks much the best. So you pick him. But that means you would pick number three in ALL races. Now your contention, if translated to this example, would be that number three will either win all races or lose all races. But this would obviously not be the case even if they were running on a straightaway with no racing luck involved.



[/ QUOTE ]

I understand your method works well for picking horses. From what I've heard you have perfected the method to such a degree that you are one of the rare successful handicapers. At least I've heard you make bets on the horses and I can't imagine you doing this over a long period of time if you weren't successful at it.

So I'm not really fundamentally opposed to the Baysian approach to probability when it has good applications. The real question is whether the court case is a good application.

A big problem in applying the approach in court cases is the way evidence is gathered. It's Not the kind of straightforward reporting of data like on the horse racing form. Police tend to find suspects and focus in on them. They then Look for circumstances that link the suspect to the crime. This means that not all evidence collected is independent. It's usually only the very initial evidence that is unbiased. What I can't imagine is how the Correlation of evidence so produced can possibly be automatically modeled. The Jury's human judgement for the relative strength of such evidence taken as a whole is vital in my opinion.

For example, in your Shoe Size thread you have people convinced that the appropriate model is 1 million identical cases with 800,000 guilty and 200,000 innocent based on pre-Shoe Size evidence. With say, 1% of the general population matching the new Shoe Size evidence you conclude the Innocent Percent now must be 2000/(2000+800,00) or about 1 quarter of 1 percent. Why do we have the feeling that something is wrong here?

The reason is because it's not an accurate model. Consider the following Model. The police investigate 1 billion similiar cases. They find pre-Shoe Size evidence in 1 million of those cases whereby 800,000 supects are guilty and 200,000 are innocent. Determined to nail the case down, the police find additional circumstances that match all 1 million suspects to the crime. The one and only Trial we are in just happens to be the one where the matching circumstance is the Shoe Size.

Not only do you not get the same Baysian conclusion for this Model but you really don't know how much of this kind of thing was happening for all the other evidence collected in the 1 Billion cases. We're just assuming the Police had 80% accuracy for that evidence. In reality this Correlation of evidence may be happening for a lot of the evidence.

Furthermore, we just happen to be able to see the flaw in your straightforward model for this situation. In general, the flaw may not be so apparent. But if we get the feeling something's wrong it's probably a good idea to pay attention to it.

PairTheBoard

[bunny]

I'm an ex-mathematician but with only a fleeting interest in probability. Your interpretation seems obvious and "standard".

I think what PairTheBoard is making reference to is the difficulty in ascribing a probability to a one-off event. The guy's either guilty or he isnt. There is a valid philosophical question as to what it means to say "The probability that event X occurred is 40%" but I think he is making too much of it in this case.

[Piers]

[ QUOTE ]
I gave a precise definition of the pseudo probability as you call it. The juror is asked to imagine that there are 100 trials with the exact same evidence. How many of those defendents IN HIS OWN PERSONAL OPINION will be innocent. As it is, the juror is expected to give the answer "not many" before he convicts. So all I am saying is that it would be nice for jurors to be told what number should be considered not many.

[/ QUOTE ]

I think the idea you are trying to express is well formed; I am not sure whether it is a good idea in practice.

Firstly, a lot of people have problems with numbers. So using a something fuzzy like “not many” is going cause less confusion than a number like 5%, or rather the people getting confused by “not many” are likely intelligent enough to sort things out for themselves.

Secondly using a number like 5% makes it clear to everyone that the legal system is designed so that 5% of the time there will be a miscarriage of justice. That’s likely to frighten a lot of people, and risk the legal system loosing the respect of the public. Using fuzzy language like almost certain allows a smokescreen to be put over the whole subject, and avoids alarming people too much.

[ QUOTE ]
I cannot imagine how there could possibly be 100 different trials with the exact same evidence. That scenario is a figment of your imagination. When I imagine 100 copies of the 1 situation that is in front of me, all I can see are either 100 guilty defendents or 100 innocent ones. I just don't know which. That is a philosophical difference in how we look at it. I am not bound by your philosophical view nor by the imaginary figment you have conjured.

[/ QUOTE ]

Possibly PairTheBoard is getting a little carried away with his contrariness. Although I cannot see anything logically invalid in his statement.

For instance, I am sure everyone agrees including you that the “100 different trials with the same evidence” is a figment of your imagination.

How I love, “the grass is green”, “no you’re wrong the sky is blue” arguments.

[PairTheBoard]

[ QUOTE ]
[ QUOTE ]
I cannot imagine how there could possibly be 100 different trials with the exact same evidence. That scenario is a figment of your imagination. When I imagine 100 copies of the 1 situation that is in front of me, all I can see are either 100 guilty defendents or 100 innocent ones. I just don't know which. That is a philosophical difference in how we look at it. I am not bound by your philosophical view nor by the imaginary figment you have conjured.

[/ QUOTE ]


Possibly PairTheBoard is getting a little carried away with his contrariness. Although I cannot see anything logically invalid in his statement.


[/ QUOTE ]

I'm not fundamentally opposed to this kind of imaginary conjuring. I realize it can work well for things like making betting odds on horse races. However, I don't know that it always applies so well. There's a Baysian philosophy that says it always applies. That's the philosophy I'm not bound by. And I'm very dubious about its application for weighing evidence in a court case.

A problem I have trying to imagine the 100 court cases with identical evidence is that I can't imagine the larger pool of data from which that evidence was gathered in each of the 100 cases. How much of that data was looked at by police, how much was ignored, and how much was filtered to fit the suspect under investigation? How might the identical evidence in the 100 cases be corrolated differently with respect to the larger pool of data? If we keep it simple and assume the total universe of data is identical in all 100 cases then we are at my point where they're either all guilty or all innocent.

There's also some question in my mind if we are misconceptualizing things at an even more basic level. I'm not sure measuring things on a scale of [0,1] is even correct. There's a weighing of evidence on the scales of justice. The defendant is presumed innocent at the beginning. Suppose evidence is presented at the beginning that's exculpatory. Is he now considered more innocent than innocent? If he starts at 0 is he now still at 0? If additional evidence is added for guilt does he now go up the scale from 0 just like he would if there was no exculpatory evidence?

The whole Sklansky approach just looks very very iffy to me. I'm not just being contrarian. I know the mathematical probability better than Sklansky does. If he would study a lot more math himself he might come to realize how tricky it can sometimes be and the kind of absurd results you can get if you're not careful. When I see as many warning lights flashing as I see in this situation I'm certainly not going to allow myself to be rushed into rash conclusions based on misleading application of dubious math models.

PairTheBoard

[PLOlover]

[ QUOTE ]
There's also some question in my mind if we are misconceptualizing things at an even more basic level. I'm not sure measuring things on a scale of [0,1] is even correct. There's a weighing of evidence on the scales of justice. The defendant is presumed innocent at the beginning. Suppose evidence is presented at the beginning that's exculpatory. Is he now considered more innocent than innocent? If he starts at 0 is he now still at 0? If additional evidence is added for guilt does he now go up the scale from 0 just like he would if there was no exculpatory evidence?

[/ QUOTE ]

The first thing I thought of was that
a) presumption of innocence (100% not guilty - 0% guilty)
b) no preconceptions/mind not made up (50% not guilty - 50% guilty)

[PLOlover]

As a non math guy let me just throw something out there.
2 data items:

1) jury - 80% sure guilty ->80% of time guy is guilty, or as in DS post 800k/1mill are guilty

2) in model of 1million defendants 800k guilty and have shoe size, 200k innocent and 2k have shoe size

-----------------------
opinion: even though it is natural to think 1) and 2) are talking about the same data set, I wonder if this natural assumption is true.

I'm not really sure how to word it, sorry. It seems to me there's some crossover between 1) and 2) that kinda begs the question or something.

----

ok how's this. let me reword stuff to accentuate my point.

1) the jury will pick correctly 8/10 times.
2) in a 1,000,00 trials the a person is found guilty 800k times.

now given my new 1) and 2), does that change the problem outcome?
Notice that now some guilty men are found not guilty
and
some innnocent men are found guilty.

----------------------------------
another point maybe here is what I was trying to think of.
Let's assume that the jury guilt percent is not a linear function.
I realize in the original DS post it was defined so I'm not sure
how relevant this is.
For example, at
99% 9.5/10 are actually guilty
95% 9/10 are actually guilty
90% 8/10 are actually guilty
80% 5/10 are actually guilty
70% 2/10 are actually guilty

-------------------------------
I guess I'm rambling sorry just throwing stuff out there

[jason1990]

Like it or not, you are sitting smack dab in the middle of what may be one of the most contentious philosophical debates of the 20th century. bunny is right that much of this centers around assigning probabilities to single events. The frequency philosophy says that probability only makes sense in the context of a long sequence of independent trials. For example, its adherents contend that it makes no sense to talk about the probability that Hillary Clinton will be the next president, since it is a one time event. On the other hand, Bayesian philosophy says that objective probabilities do not exist. According to them, the probability that Hillary Clinton will be the next president does not exist in any objective sense. You can talk about it, but only by asserting your own subjective view of the matter. All subjective views are equally valid. If you think the brand new quarter in my pocket has probability 1/3 of landing heads, then that is your view. Who am I to argue? In Bayesian philosophy, all that matters is that your subjective views are consistent, so that a Dutch book cannot be made against you. This debate relates to the jury discussion because an individual murder is a one-time event. Frequentists say probability concepts do not apply. Bayesians say they apply, but it is all subjective. Any consistent opinion on the matter is as valid as any other.

In my opinion, both philosophies are mostly useless. The Bayesian philosophy, taken literally, is absurd. For instance, if there is physical symmetry in a system, such as the rolling of a symmetric die, I am convinced that, at least approximately, all sides are equally likely and that this is an objective statement. The symmetry of the fair coin tells us, objectively, that the probability of heads is (at least approximately) 1/2. A Bayesian who says otherwise is deluded.

Frequentists can be just as ridiculous. Imagine this: I have a deformed coin. I am about to flip it twice. I will destroy it after two flips. I claim that the probability of flipping heads first, tails second is the same as the probability of flipping tails first, heads second. This claim is based on symmetry in time and I consider it an objective statement of fact. But a frequentist would tell me my claim is meaningless. Since the coin will be destroyed, there is no long run sequence, so the concept of probability does not apply.

Regarding the horses, I am imagining a scenario in which you come to me with the racing form and you ask me, "what is the probability horse 3 will win?" I would probably try to build a model and come up with a number for you. If pressed, I would freely acknowledge that this number represents my subjective opinion. But I would try to argue that it is a "good" opinion by appealing to whatever facts I uncovered and incorporated in my model.

On the other hand, I might just answer you by saying, "I don't know." I think this is a legitimate answer and, in this case, is probably the only completely objective answer. If I was intent on remaining 100% objective, I would simply refuse to answer your question. You might then ask, "do you believe, beyond a reasonable doubt, that horse 3 will win?" I think I could answer that question without deciding on a specific numeric probability. You might even ask, "do you think there is a preponderance of evidence indicating that horse 3 will win?" I think I could also answer that question without deciding on a specific numeric probability.

In other words, if I met a juror who refused to assign a numeric value to the probability of guilt, and justified it by claiming a desire to remain as objective as possible, then I would consider that a rational stance and I would not be concerned that this stance, in and of itself, would prevent him from doing his job as a juror.

But if met someone who refused to assign a numeric value to the probability that I win my next bet at the roulette wheel, and justified it by claiming a desire to remain as objective as possible, then I would consider that person to be in denial about the practical reality of probability.