Are most scientific studies screwed up?

[Rduke55]

Agreed about Science and Nature. Some of the papers in there are amazingly crappy.
Funding is pretty screwed up. I think the publishing and funding have the same problem. You have these dinosaurs taking the lion's share (is that mixing metaphors?) of publishing and funding and acting as gatekeepers.
A recent bar conversation we've been having is how to fix it.
How would you suggest fixing it? I'm pretty evenly split between public and private funding and they are both screwed up (maybe for different reasons).

[Borodog]

[ QUOTE ]
Agreed about Science and Nature. Some of the papers in there are amazingly crappy.
Funding is pretty screwed up. I think the publishing and funding have the same problem. You have these dinosaurs taking the lion's share (is that mixing metaphors?) of publishing and funding and acting as gatekeepers.
A recent bar conversation we've been having is how to fix it.
How would you suggest fixing it? I'm pretty evenly split between public and private funding and they are both screwed up (maybe for different reasons).

[/ QUOTE ]

You know how I would suggest fixing it, and you probably wouldn't like my suggestion. Whenever funding is allocated politically, those funding decisions will be based on politics, and not on economic efficiency OR good science.

[Borodog]

Also, I was hoping to hear that the crappiness of Science and Nature was just limited to astrophysics. That's actually disappointing to hear that it is more widespread. [img]/images/graemlins/frown.gif[/img]

[Rduke55]

[ QUOTE ]
[ QUOTE ]
Agreed about Science and Nature. Some of the papers in there are amazingly crappy.
Funding is pretty screwed up. I think the publishing and funding have the same problem. You have these dinosaurs taking the lion's share (is that mixing metaphors?) of publishing and funding and acting as gatekeepers.
A recent bar conversation we've been having is how to fix it.
How would you suggest fixing it? I'm pretty evenly split between public and private funding and they are both screwed up (maybe for different reasons).

[/ QUOTE ]

You know how I would suggest fixing it, and you probably wouldn't like my suggestion. Whenever funding is allocated politically, those funding decisions will be based on politics, and not on economic efficiency OR good science.

[/ QUOTE ]

Well, that's why I threw in that private funding was screwy too. Economic efficiency and funding good science are not things I think of for either foundations or biotech. It may be a result of the people running this and some atrocious communication but they're pretty messed up (I may be biting the hand that feeds me here [img]/images/graemlins/smile.gif[/img] ).

I think the mechanism in place at the NIH is pretty solid and could be saved with some changes. It does have scientists making the decisions as to where the money goes and I think when compared to other funding agencies, including private, it's the best we have. Changing review panel compositions, etc. and some other aspects of the process would do wonders IMO.

One thing that the Nat'l Institute for Aging in the NIH has done is make it a point to really focus on funding newer investigators and I think this is paying off and will pay off bigtime.

[Rduke55]

[ QUOTE ]
Also, I was hoping to hear that the crappiness of Science and Nature was just limited to astrophysics. That's actually disappointing to hear that it is more widespread. [img]/images/graemlins/frown.gif[/img]

[/ QUOTE ]

Well, they certainly had their crap moments in neuroscience. Journals are funny in that changing just an editor or two can really change things dramatically. There are some other pretty good journals that have been going downhill recently (and others that have improved, too)

Add the fact that Science and Nature have a huge impact factor, extreme restrictions on space and figures, and enormous popularity with CNN, NYT, etc. and I don't see how you avoid them pandering and turning into the People magazine of science.