The envelope problem, and a possible solution

[AaronBrown]

[ QUOTE ]
I can see that you want to use the Paradox as a vehicle for discussing the (important) philosophical issues at the heart of statistical inference. In my opinion, this is an excellent use of the Paradox. But rather than coming right out and saying that is what you are doing, you phrase things in a way that can lead the less informed to believe that the branch of mathematics which we call probability theory is logically inconsistent.

[/ QUOTE ]
Oh, sorry, thank you for putting it so clearly. I should have said this at the beginning. I believe the paradox was invented and discussed by people interested in the application of probability theory to real problems. No one disputes that you can get a consistent theory of probability, the open issue is whether that consistent theory is rich enough to answer useful questions.

In fact, we have not one, but four consistent theories of probability, due to Shannon, Arrow-Debreu, Savage and Von Neumann. Savage was a great Bayesian, Shannon was the most objectivist (his theory of probability is popular in quantuum mechanics, possibly the only true physical randomness). Shannon's the guy who built a mechanical hand to flip coins reliably heads or tails (Ed Thorp saw it and immediately made the connection to roulette, the two of them built wearable computers to predict spins in a casino). The other two proposed abstract probabilities that were objective in principle but required conceptual experiments to define. No doubt there are other consistent theories as well, that never attacted major followings.

I agree that the envelope paradox uses standard mathematical tools to arrive at two opposite conclusions. Mathematics must be consistent, or you can prove everything, so we have to rule out one or the other chain of logic. In order to do that, we have to rule out calculations that are used every day in statistics. Statistics can survive these radical surgeries, but I think they are foolish. Bayesians reject useful tools, Frequentists engage in tedious hair-splitting. I'd rather do the math the natural way and trust that someone will figure out how to make it rigorous someday. Applied mathematical practice has always been ahead of theory, and is usually (but not always) justified in the end.

The batting average question is also pure mathematics. The question isn't about real baseball players. It asks: What is the probability that two automotons with constant, independent and equal probabilities of getting hits would differ in batting average as much as the two players in question? The answer is the same if the automotons flip coins or spin roulette wheels, only the probabilities matter. The use of this answer in inference, of course, does involve baseball.

I also agree that the paradox does not specify mathematical reasoning at all, and that is key. It asks a common sense question that appears to have two opposing common sense answers. People layer the mathematics on to it. That's fine, but making technical assumptions to make one argument right and one wrong doesn't answer the fundamental question of the paradox.

My beef is not with people who make one set of assumptions or another, although I prefer to live with inconsistency. My beef is with people who insist mathematics dictate the choice of assumptions. That's wrong. There is no reason outside of personal preference to prefer one approach over the other.

Consider the analogy to Godel's famous result of that mathematics cannot be both consistent and complete. He makes some assumptions, such as that mathematics includes counting numbers. Some people have tried to rescue a complete consistent mathematics by eliminating counting numbers. That might be a useful intellectual exercise (probably not) but it clearly takes mathematics far away from what people want to use it for. Most mathematicians accepted the result and decided to live with an incomplete mathematics, in which there were true, false and undecidable statements. The universe survived.

[PairTheBoard]

[ QUOTE ]
[ QUOTE ]
You asked for something that makes common sense. How much common sense does your statement make?

[/ QUOTE ]
Remember, I've said from the beginning that there is no common sense answer.

At this point, I think we should just agree that your common sense is broader than mine. You can accept probabilities that change when you look at them, as a Bayesian does, and also mathematical formulae that are different depending on whether X is a number or a variable, as a Frequentist does.

I'd rather reject both and live with having no rigorous theory of probability that can also answer useful questions. Many people agree with me, many others choose the less distasteful (to them) of the two alternatives above. But you are rare in being willing to embrace both. For you, the envelope problem is no problem at all.

That probably sounds sarcastic, and maybe I meant it that way a little, but I admit I have no argument against it. You might well be right, I just hope you aren't.

[/ QUOTE ]


So would you take part 1 and part 2 in the Proposition Bet below? If so why and if not why not?

Look at this from a Gambler's point of view. You are told about the two envelopes and asked to choose one. You are offered these Proposition Bets.

1. Given 3-2 odds, would you be willing to bet $10 that your envelope contains the smaller amount?

2. Given 3-2 odds, would you be willing to bet the amount in your Envelope that it is the smaller amount?


1 is a good bet for you while 2 is not. Do you see why there is a difference between the two?


PairTheBoard

[AaronBrown]

[ QUOTE ]
So would you take part 1 and part 2 in the Proposition Bet below? If so why and if not why not?

1. Given 3-2 odds, would you be willing to bet $10 that your envelope contains the smaller amount?

2. Given 3-2 odds, would you be willing to bet the amount in your Envelope that it is the smaller amount?

1 is a good bet for you while 2 is not. Do you see why there is a difference between the two?

[/ QUOTE ]
I'm not personally insulted at being asked this question, but asking it does seem to imply anyone who thinks the envelope paradox raises important questions doesn't understand simple probability concepts. Is that your intent?

There are people who are genuinely confused about whether or not to switch envelopes, who might think the two bets above are the same. But it takes only logic, not probability theory, to see that the bet in (2) can be converted to an obviously poor bet of winning half the total amounts in the two envelopes if you have the smaller amount and paying two thirds of the total amount if you have the larger amount. That's not a mystery to me or anyone who thinks through it clearly.

The challenge is to come up with a consistent way of computing which bets to take. It's a lot harder than it looks. The envelope paradox is simplified to make the basic contradictions clear, which makes it easy to "resolve" with ad hoc techniques. But those techniques are not much practical help in realistic problems.

[PairTheBoard]

[ QUOTE ]
There are people who are genuinely confused about whether or not to switch envelopes, who might think the two bets above are the same. But it takes only logic, not probability theory, to see that the bet in (2) can be converted to an obviously poor bet of winning half the total amounts in the two envelopes if you have the smaller amount and paying two thirds of the total amount if you have the larger amount. That's not a mystery to me or anyone who thinks through it clearly

[/ QUOTE ]

So there's no mystery with the proposition bet? It doesn't produce a true paradox with equally strong arguments on both sides for why it might be a good bet? All it takes is simple logic to see why it's not a good bet? Yet the Envelope Paradox is essentially the same situation. Only in the Envelope "Paradox" the Proposition asks if you want to bet half the amount in your envelope giving you 2-1 odds.


PairTheBoard

[jason1990]

I'm still a little bit confused about your stance here. Let me clarify some things from my side first. When I say "probability theory", I am talking about the branch of mainstream measure theory that studies sets, sigma-algebras on those sets, and countably-additive, non-negative measures on those sigma-algebras whose total mass is 1. This approach to probability theory was introduced by Kolmogorov. Perhaps you could elaborate on the four theories you mentioned, with links if possible, and explain their relationship to what I am calling "probability theory".

You said,

[ QUOTE ]
I agree that the envelope paradox uses standard mathematical tools to arrive at two opposite conclusions. Mathematics must be consistent, or you can prove everything, so we have to rule out one or the other chain of logic. In order to do that, we have to rule out calculations that are used every day in statistics ... My beef is with people who insist mathematics dictate the choice of assumptions. That's wrong. There is no reason outside of personal preference to prefer one approach over the other.

[/ QUOTE ]
I still don't understand what these two "approaches" are. Here's one approach: we assume that
<ul type="square">
[1] if A and B are events, then P(A | B) = P(A and B)/P(B),
[2] if A and B are independent events, then P(A | B) = P(A),
[3] if A and B are mutually exclusive events, then P(A or B) = P(A) + P(B),
[4] if A is an event, then 0 &lt;= P(A) &lt;= 1.[/list]As far as I can tell, if we assume these four things, then it cannot be the case that

P(Y = 2X | X = k) = 0.5

for all k.

What is the other approach? Which of these assumptions does it reject? Does it involve one of these other "alternative" probability theories you mentioned? Because as far as I can tell, the work of Shannon and Arrow-Debreu was within the context of Kolmogorov's formalism, and Savage's main departure from the formalism was his use of "measures" which were only finitely additive. But the above assumptions do not require countable additivity, nor do they even require sigma-algebras, only algebras. You also suggest that this other approach, whatever it is, is used every day in statistics. So are you saying that there are mainstream statisticians who reject one or more of the above assumptions on a daily basis?

[jason1990]

[ QUOTE ]
[ QUOTE ]
I agree that the envelope paradox uses standard mathematical tools to arrive at two opposite conclusions. Mathematics must be consistent, or you can prove everything, so we have to rule out one or the other chain of logic. In order to do that, we have to rule out calculations that are used every day in statistics ... My beef is with people who insist mathematics dictate the choice of assumptions. That's wrong. There is no reason outside of personal preference to prefer one approach over the other.

[/ QUOTE ]
I still don't understand what these two "approaches" are. Here's one approach: we assume that
<ul type="square">
[1] if A and B are events, then P(A | B) = P(A and B)/P(B),
[2] if A and B are independent events, then P(A | B) = P(A),
[3] if A and B are mutually exclusive events, then P(A or B) = P(A) + P(B),
[4] if A is an event, then 0 &lt;= P(A) &lt;= 1.[/list]As far as I can tell, if we assume these four things, then it cannot be the case that

P(Y = 2X | X = k) = 0.5

for all k.

What is the other approach? Which of these assumptions does it reject? Does it involve one of these other "alternative" probability theories you mentioned? Because as far as I can tell, the work of Shannon and Arrow-Debreu was within the context of Kolmogorov's formalism, and Savage's main departure from the formalism was his use of "measures" which were only finitely additive. But the above assumptions do not require countable additivity, nor do they even require sigma-algebras, only algebras. You also suggest that this other approach, whatever it is, is used every day in statistics. So are you saying that there are mainstream statisticians who reject one or more of the above assumptions on a daily basis?

[/ QUOTE ]
The above quote is from my last post. After I posted it, I thought a little more and I think you already told us what the other approach is. You said,

[ QUOTE ]
I'd rather do the math the natural way and trust that someone will figure out how to make it rigorous someday. Applied mathematical practice has always been ahead of theory, and is usually (but not always) justified in the end.

[/ QUOTE ]
So the first approach is the one I outlined above which tells us that the conditional probability of having the larger envelope, given that we observe k in the chosen envelope, cannot be 0.5 for all k.

The second approach is to have no formalism at all. To just do what feels right, and worry about formal justification later. This is the approach that tells us we should always switch, regardless of what is in the envelope.

In the second approach, not only can we not justify the steps, but they contradict assumptions [1]-[4]. And we are led to a conclusion which is intuitively absurd. But the method is so natural, it must be right, and someone will probably be able to rigorously justify it in the future. Moreover, there is no reason outside of personal preference to prefer one approach over the other.

If I met a statistician who thought this way, I wouldn't trust him with my piggy bank. I really hope that I'm wrong and that the "other approach" has something to do with Von Neumann.

[jason1990]

[ QUOTE ]
it takes only logic ... to see that the bet in (2) can be converted to an obviously poor bet ... That's not a mystery to me or anyone who thinks through it clearly.

The challenge is to come up with a consistent way of computing which bets to take. It's a lot harder than it looks. The envelope paradox is simplified to make the basic contradictions clear, which makes it easy to "resolve" with ad hoc techniques. But those techniques are not much practical help in realistic problems.

[/ QUOTE ]
I am stubbornly sticking to my prior my belief about you, Aaron, which is that you are an educated and intelligent man who can communicate effectively via the written word. That's why I've got to believe that there is some gross misunderstanding going on here.

In both the Paradox and in PTB's Prop Bets, the only techniques we need to use in order to arrive at the "sensible" conclusions are the techniques of elementary probability, such as assumptions [1]-[4] of my previous post. These are consistent, they are certainly not ad hoc, and they are extremely helpful in a wide array of realistic problems. Are you denying this? What am I missing here?

[AaronBrown]

[ QUOTE ]
So there's no mystery with the proposition bet? It doesn't produce a true paradox with equally strong arguments on both sides for why it might be a good bet? All it takes is simple logic to see why it's not a good bet? Yet the Envelope Paradox is essentially the same situation. Only in the Envelope "Paradox" the Proposition asks if you want to bet half the amount in your envelope giving you 2-1 odds.

[/ QUOTE ]
I'm not sure why you keep belaboring this. I know the envelope switch is not a good bet, that's not the point. I don't believe that a friend and I can make each other infinitely wealthy by passing two envelopes back and forth all day (I tried, it doesn't work). You can, however, have two people get positive EV from opposite sides of a zero sum bet, if you tradie options with people who keep score in different currencies. But that's not the aspect of the paradox we've been discussing.

I think the hard part is to come up with a rigorous theory that spells out exactly when switching makes sense, that is also flexible enough to use for practical decision problems. You don't think that's hard, because you will accept things that I feel cannot be true and that are devilishly hard to apply. You're not alone in that, many people agree with you (although few are quite as broadminded as you, most limit themselves to one hard-to-swallow pill).

None of this makes me think you don't know basic logic or math, I don't know why you think that of me. If I were to get ad hominem, I'd bet you haven't done a lot of real world statistical analysis. Not because I think you've shown yourself incapable of it, but because the day-to-day experience of trying to apply your principles leads many practitioners to become agnostic like me.

[AaronBrown]

[ QUOTE ]
I'm still a little bit confused about your stance here. Let me clarify some things from my side first. When I say "probability theory", I am talking about the branch of mainstream measure theory that studies sets, sigma-algebras on those sets, and countably-additive, non-negative measures on those sigma-algebras whose total mass is 1. This approach to probability theory was introduced by Kolmogorov. Perhaps you could elaborate on the four theories you mentioned, with links if possible, and explain their relationship to what I am calling "probability theory".

[/ QUOTE ]
That's easy enough. You are giving Kolmogorov credit for what I called Von Neumann's formulation. Many people contributed to all four systems, I didn't mean to exclude anyone, I just used familiar tag-lines.

Jimmy Savage's The Foundation of Statistics not only gives his formulation, it has clear explanations and contrasts with the other three, plus an extensive annotated bibliography.

Everyone uses sets, sigma-algebras and measures. But there is controversy over concepts like shrinkage and resampling, which do not adapt well to those tools.

[AaronBrown]

[ QUOTE ]
I still don't understand what these two "approaches" are. Here's one approach: we assume that
<ul type="square">
[1] if A and B are events, then P(A | B) = P(A and B)/P(B),
[2] if A and B are independent events, then P(A | B) = P(A),
[3] if A and B are mutually exclusive events, then P(A or B) = P(A) + P(B),
[4] if A is an event, then 0 &lt;= P(A) &lt;= 1.[/list]As far as I can tell, if we assume these four things, then it cannot be the case that

P(Y = 2X | X = k) = 0.5

for all k.

What is the other approach? Which of these assumptions does it reject? Does it involve one of these other "alternative" probability theories you mentioned? Because as far as I can tell, the work of Shannon and Arrow-Debreu was within the context of Kolmogorov's formalism, and Savage's main departure from the formalism was his use of "measures" which were only finitely additive. But the above assumptions do not require countable additivity, nor do they even require sigma-algebras, only algebras. You also suggest that this other approach, whatever it is, is used every day in statistics. So are you saying that there are mainstream statisticians who reject one or more of the above assumptions on a daily basis?

[/ QUOTE ]
I did not mean to suggest that any of the five mathematicians above believe in switching envelopes. I mentioned them only to demonstrate we do not have consensus over the best mathematical way to handle probability. It's not that these camps think they have found mathematical errors in the others, it's that they think they have mathematics that correspond better to the natural idea of probability, which predates mathematical formalism.

I accept all four of your assumptions, and don't know anyone who does not. But how do they lead to the conclusion?

The only problem I know of with P(Y = 2X | X = k) = 0.5 for all k is that the expected value of X must be infinite. I understand there are technical complexities with dealing with distributions that allow this statement to be true, but that's not the same as proving it false. Even if you outlaw them entirely, that's only a formal resolution of the paradox, not a refutation of the force of its argument.

The two common approaches to resolution are to deny that the probabilities are equal, as you do, and to forbid computing the expected value of an unknown quantity, as Kolmogorov would.