15, damn, 10...
2005-07-14 15:29:51.03729+00 by
Dan Lyke
3 comments
Not only are 47% of all statistics made up, 16% of widely published medical studies are later shown to be bunk, and another 16% get shown to have weaker results. The abstract of the paper, published in JAMA, says that:
All original clinical research studies published in 3 major general clinical
journals or high-impact-factor specialty journals in 1990-2003 and cited
more than 1000 times in the literature were examined.
and that:
Of 49 highly cited original clinical research studies, 45 claimed that
the intervention was effective. Of these, 7 (16%) were contradicted by
subsequent studies, 7 others (16%) had found effects that were stronger
than those of subsequent studies, 20 (44%) were replicated, and 11
(24%) remained largely unchallenged.
Via /..
[ related topics:
Politics Health
]
comments in ascending chronological order (reverse):
#Comment Re: made: 2005-07-15 15:32:25.997189+00 by:
petronius
[edit history]
Seems to me that this study is not showing problems with the research so much as an artifact of the publishing and research cycle. Dr. Benway does a small, pioneering study in some new area and discovers that drug X127 has a great effect in the dreaded Barber's Itch. His study seems to be OK, so it is published so that others can A) setup their own studies or B) begin treating suffering humanity. As many people begin to run their own studies Benway's work gets heavily cited, thus entering the JAMA's list.
Now that many centers are running studies on X127, the addition of larger numbers of subjects gives us a more refinied look at the matter. We may find that by sheer happenstance Benway found the only 20 people in North America who will be helped by X127, while everybody else develops near-fatal chillblaines. Or that when we have a wider sample and factor in age, ethnicity, underlying conditions etc, X127 only helps about 53% of chronic itchers. We would never have any of this information without the original paper, and without Benway's first attempt nobody would bother looking at the subject.
66% of the studies support the original information, and another 16% shows an effect, albeit lessened. Sounds pretty good to me. Of course, this is only the first study. Maybe a look at more than 45 papers will relegate this study to the contradicted 16%.
#Comment Re: made: 2005-07-15 15:52:30.58448+00 by:
ebradway
The one statistic that seems to be avoided so often in medical publications is just how effective the "placebo" pill is. The placebo effect actually has a higher statistical success rate than many commercial medications.
So just what would have happenened had Neo taken the placebo (Blue Pill)?
I also think we should start bottling M&Ms with a big "Placebo" on the label. We could make all kinds of health claims based on the statistically proven effectiveness of the placebo effect. All we'd have to do is add a disclaimer that the medicine only works if you beleive it will.
#Comment Re: made: 2005-07-15 21:04:08.260184+00 by:
petronius
The health claims we make for "Placebo" can't be any more inflated than those for various vitaimin supplements and natural nostrums I see hawked on TV and get in the mail. If a real pharma company made claims a tenth as strong they'd be sharing a cell with Bernie Ebbers.