[JaredL]

Discussions like this make me wish I did applied work and not theory. I have a couple half-baked ideas about education, with datasets in mind, but I have to work on other stuff until at least I go on the market next year.

Econophile is going to own this thread. This is now obvious from his posts, but was also pretty obvious just from reading the OP.

The problem with this discussion is that while people are posing good questions they are pretty much all going to fall into two categories:
1. Questions that have been answered but didn't make Freakonomics or other major media outlets.
2. Those that economists or others have also considered but for which there is no reasonable data set.

Data sets that answer causality are extremely hard to come up with. Generally you need some variance from sources outside the system otherwise you can't untangle the correlation.

For example, take MrMon's suggestion of looking at teacher pay and student success. The interesting question here would be something like "What is the effect of increasing/decreasing the salary of public school teachers on the success of students in their school?"

The problem here is that districts that pay more money tend to contain people that are wealthy. People who are wealthy tend to be well educated. Wealthy and well educated parents tend to have children that are more successful in schools. So the problem here is that there are factors, parents' wealth and education, that are actually driving both the pay and success of students. Unless you can find a dataset that does not have this problem you can't answer the question. This sometimes but not often the case.

I don't do empirical stuff so I'll be less biased than others but I agree with OP's friend. I find it unlikely that the questions people will bring up here have not been thought up by researchers. The real key to these things is finding the right model and data that answer the question well. That is hard.