Ok, so we have the
No Free Lunch Theorem (NFLT). I can sort of understand this intuitively. However, while reading an article the other day in The New Yorker, I saw the following example:
"Consider a search for high ground on some unfamiliar, hilly terrain. You’re on foot and it’s a moonless night; you’ve got two hours to reach the highest place you can. How to proceed? One sensible search algorithm might say, “Walk uphill in the steepest possible direction; if no direction uphill is available, take a couple of steps to the left and try again.” This algorithm insures that you’re generally moving upward. Another search algorithm—a so-called blind search algorithm—might say, “Walk in a random direction.” This would sometimes take you uphill but sometimes down. Roughly, the N.F.L. theorems prove the surprising fact that, averaged over all possible terrains, no search algorithm is better than any other. In some landscapes, moving uphill gets you to higher ground in the allotted time, while in other landscapes moving randomly does, but on average neither outperforms the other."
This does not come across as intuitive for me. To me, it seems that if you choose a step up instead of a step down, on average, you would find higher ground. Though, it seems to be a pretty straightforward application of the NFLT. Any thoughts?