Ockham's Razor

[PairTheBoard]

The thing is, for any randomly generated program that computes pi correctly for some digits, that program can have add-on lines of code that don't essentially change it. It can have infinitely many add-ons. I don't see any way you could reasonably Pronounce a probalility measure on that space of unbounded finite randomly generated codes. I don't think the proper measure on such a space would even be a probability measure.

If this arguement really has technical merit, somebody has written a peer reviewed paper on it somewhere that you should be able to reference. I doubt you can do that.

PairTheBoard

[luckyme]

[ QUOTE ]
The thing is, for any randomly generated program that computes pi correctly for some digits, that program can have add-on lines of code that don't essentially change it. It can have infinitely many add-ons. I don't see any way you could reasonably Pronounce a probalility measure on that space of unbounded finite randomly generated codes. I don't think the proper measure on such a space would even be a probability measure.

If this arguement really has technical merit, somebody has written a peer reviewed paper on it somewhere that you should be able to reference. I doubt you can do that.

PairTheBoard

[/ QUOTE ]

Have you not mixed 'what it is doing' with 'what it is'?

luckyme

[jason1990]

Here is an article on algorithmic probability: http://www.scholarpedia.org/article/...ic_Probability . I do not know anything about it. But the article seems to be saying (near the beginning) that the choice of the prior distribution in algorithmic probability is justified, in part, by Occam's razor.

[Piers]

[ QUOTE ]
None of which effects the result that simpler programs are the more likely explanation, given access to a finite amount of the output (and the assumption of truly random input -- i.e. no cheating allowed).

"Your computer will eventually burn up" is not really a good objection to rigorous results concerning algorithmic probability.

[/ QUOTE ]

The point I am making is that although Ockham's Razor is an extremely useful tool for practical applications, it is guaranteed to give an inaccurate model in any non trivial situation.

You can restrict consideration of the program results to algorithmic consideration if you want, but reality respects no such bounds.

[ QUOTE ]
(and the assumption of truly random input -- i.e. no cheating allowed).


[/ QUOTE ]

You cannot reliably make such assumptions in a real situation, only in a toy model.

[ QUOTE ]
"Your computer will eventually burn up" is not really a good objection to rigorous results concerning algorithmic probability

[/ QUOTE ]

In your own model maybe, but a real computer does not exist inside your model. The program being run is a guide to what will happen next not a guarantee. A real computer will not continue churning out digits of pi forever. Something will happen, if it’s like my PC the CPU will likely overheat at some point and the OS will crash [img]/images/graemlins/frown.gif[/img]

You can model the situation as a computer continuingly outputting digits of pi. This will usually give the correct result, but it is guaranteed to fail at some point, which is the usual fate of the consequences of Ockham’s razor.

[luckyme]

[ QUOTE ]
You cannot reliably make such assumptions in a real situation, only in a toy model.

[/ QUOTE ]

It looked like a bear.
It looked like it was running at me.
It sounded like it was runninng at me.

It fell in a hole.
Therefore it wasn't a bear running at me?

Ok, you've totally confused me. If you think you can clarify enough, please do.

luckyme

[Piers]

Here is a reply I made when someone made the same post on RPG about five years ago.

[ QUOTE ]
Take a problem consider the set of explanations. Order this set by
complexity and goodness (how well it fits in with the rest of observed
reality)


The number of 'bad' solutions increases dramatically with increase in
complexity, hence if you pick two solutions, the simplest is
almost always going to be better. This is basis for Occam's
Razor.


Paradoxically it is also true that for any solution, every higher
level of complexity contains a 'better' solution. This does not
contradict Occam's Razor, just highlights something different.




[/ QUOTE ]

[luckyme]

I'm trying to follow this exchange and seem to be reading a mixture of two different concepts ( or I'm just missing the points as OR suggests :-)

Those familiar with Dennetts work may see the jumping between the design stance and the physical stance.

To use the computer pi example. It appears to be coming from a program "designed" to produce pi and that is the OR position.
It doesn't matter whether the program is running on strung together rice crispies or a Cray and will one day be subject to mice gnawing on key parts and muck up the output.
To mix the claim about the programs output, which is the level OR is dealing with in this example with the physical makeup of what the program is running on is shifting to a different topic altogether.
That level would apply if OR was making a statement about the physical structure of the system producing the output.

that's the best I can to to explain my confusion,

hope there is help out there, luckyme

[Borodog]

[ QUOTE ]
Ockham's Razor is a useful rule of thumb for simplifying decision-making.

But however useful Ockham’s razor might be in practise it is instructive to observe that it is almost always wrong. There will always be a more complicated model that is more accurate, but which is likely lost amongst the uncountable number of plausible but incorrect more complicated models.

[/ QUOTE ]

That doesn't make Occam's Razor wrong. Do you see why?

[Piers]

[ QUOTE ]
it is instructive to observe that it is almost always wrong.

[/ QUOTE ]
Ops! In an attempt to be contentious I was being a little wayward in my use of language. What I meant of course was that not that Ockham's razor itself was wrong, but any theory it promotes was, in the sense that in any non trivial scenario the will always be a more accurate theory.

[Piers]

[ QUOTE ]
Suppose you have a computer producing a string of output characters -- you don't know the program (input), but you do so far have the first 183,000 output

[/ QUOTE ]

I took this to refer to a ‘computer producing a string of output characters’, perhaps if the OP had referred instead to an ‘algorithm producing a string of output characters’ there would be less opportunity for pedantry.

Due to our limited minds, when we consider the real world we have to simplify, put boundaries around things and ignore irrelevant information. While this works in practise most of the time, the flaws are obvious.

Say we use Ockham’s razor to create a simplified model of some aspect of reality. Then create a problem within that model, use Ockham’s razor to solve that problem accurately, and claim that vindicates Ockham’s razor, feels to me a bit like circular reasoning.