Robert Matthews writes:
“>This has just appeared in BMJ Evidence Based Medicine. It addresses the controversial question of whether lowering LDL using statins leads to reduced mortality and CVD rates.
The researchers pull together 35 published studies, and then assess the evidence of benefit – but say a meta-analysis is inappropriate, given the heterogeneity of the studies. Hmmm, ok, interesting….But what they do instead is astounding. They just tot up the numbers of studies that failed to reach statistical significance, and interpret each as a “negative” study – ie evidence that the LDL reduction had no effect. They ignore the point estimates completely.
They then find that only 1 of the 13 (8%) of the studies that achieved an LDL target showed statistically significant benefit in mortality risk, and 5 of the 13 (39%) showed the same for CVD. They do the same for those studies that failed to reach the LDL target.
Incredibly, this leads them to conclude: “The negative results [sic] of numerous cholesterol lowering randomised controlled trials call into question the validity of using low density lipoprotein cholesterol as a surrogate target for the prevention of cardiovascular disease”.
The paper has several other major flaws (as others have pointed out). But surely the stand-out blunder is the “Absence of evidence/evidence of absence” fallacy. I cannot recall seeing a more shocking example in a “serious” medical journal. Whatever the paper does call into question, top of the list must be how this came to be published – seemingly without irony – in BMJ Evidence Based Medicine.
If nothing else, the paper might serve as useful teaching material..
P.S. For what it’s worth, even a simple re-analysis focusing solely on the point estimates produces a radically different outcome.
BTW, for some reason, the authors have included studies whose mortality benefit is stated as “NR” (Not Reported) in their tables.
Also, this means the percentages used to calculate their bar charts are also incorrect (eg of those studies that met their LDL reduction targets, there are only 10, not 13, studies that allow the mortality benefit to be calculated).
“British Medical Journal Evidence Based Medicine,” huh? Remember our earlier post, Evidence-based medicine eats itself.
Evidence Based Medicine is more than a slogan. It’s also a way to destroy information!