Under the subject line, “Null misinterpretation of CIs reaches new level of lethality,” Sander Greenland points us to this article with the following in the Results section:
Compared to no masks there was no reduction of influenza-like illness (ILI) cases (Risk Ratio 0.93, 95%CI 0.83 to 1.05) or influenza (Risk Ratio 0.84, 95%CI 0.61-1.17) for masks in the general population, nor in healthcare workers (Risk Ratio 0.37, 95%CI 0.05 to 2.50).
Sander writes, “These results will likely be cited by authorities who have been trying to deny needs for masks. Seems impossible for authors to report that their results were actually too imprecise to establish an effect direction although the estimates were in the expected protective directions.”
I agree with Sander that the above quote is wrong. If you want to say there was a reduction but it was not statistically distinguishable from chance, then say there was a reduction but it was not statistically distinguishable from chance. Don’t say there was no reduction.
On the plus side, the article continues:
That time they got it right.
This illustrates one of the challenges of statistical communication: there are so many opportunities to garble the message. And even if you make no mistakes, people can still misinterpret what you’ve written, given the norm of acting as if every study either makes a discovery or proves that something equals zero.
We wrote about a very similar story last month.