Opinion: Were masks useless? The deceptive interpretation of what science tells us

Editor’s Note: Kent Sepkowitz is a physician and infectious disease expert at Memorial Sloan Kettering Cancer Center in New York. The views expressed in this commentary are his own. View more opinion on CNN.

Three years after the Covid-19 pandemic changed the lives of all Americans, we are witnessing a retrospective questioning of whether masks—and mask mandates—decrease the risk of Covid-19 spread.

To try to clarify the issue, the venerable Cochrane Library led a meta-analysis of available data. Meta-analysis often is used to try to help reconcile conflicting evidence, though the approach has its doubters.

The premise is that, using a strict approach to determine which previous studies are clear enough for inclusion, experts can cobble together the results from years of work done by different investigators and published in different journals. Then, using the bigger statistical punch provided by the many, they can find a significant conclusion not evident in the few.

Cochrane’s study-of-studies, combined with its prestige, has re-animated the pens of a troupe of anti-mask-mandaters including current New York Times columnist Bret Stephens and former New York Times reporter John Tierney.

The “money quote” comes from an interview that esteemed data scientist Dr. Tom Jefferson, the lead author of the Cochrane study, gave in an interview: “There is just no evidence that (masks) make any difference, full stop.” If his statement were an accurate reflection of what the article found, the launching of a thousand op-eds would be understandable.

But this emphatically is not the conclusion of the twelve authors, of which Jefferson was just one voice. The actual Cochrane article concludes something directly at odds with Jefferson’s statement: “The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions… The low to moderate certainty of evidence (in favor of masks) means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect,” the article notes.

In plain English, this, at least to me as a veteran reader of these types of reviews, translates to: The primary studies are not strong enough for us to say much of anything except that we may be wrong no matter what we say.

A lead author such as Jefferson is of course free to speak his mind but, at least in the interview, he apparently did not distinguish his personal view from the consensus view of the authors. There may be a back-story to this: A professor at Oxford University, Jefferson seems to long have had his doubts about masks, writing articles in The Spectator and other journals to air out his sometimes curmudgeonly views.

Given their awareness of his strong personal opinions on the topic, the Cochrane Library, in the section of the recent meta-analysis called “Authors’ Declarations of Interest” writes the following: “TJ (Tom Jefferson): reports declaring an opinion on the topic of the review in articles for popular media. TJ is an Editor at the Cochrane Acute Respiratory Infections group but was not involved in the editorial process for this review. See full statement here: https://restoringtrials.org/competing-interests-tom-jefferson/.”

Though half of the 12 authors were “not involved in the editorial process,” for this particular article, the additional characterization by Cochrane of Jefferson, the opinionated citizen (“declaring an opinion on the topic … for popular media”), is quite unusual for the medical literature.

Furthermore, several meticulous experts active on social media have unpacked every square inch of the article to reveal many possible flaws in the analysis itself that may have blunted evidence of mask efficacy. This sort of back and forth adopts the tone of “my expert is bigger than your expert” and never leads to any resolution of differences.

Personalities and spokespeople aside, there may be an even larger issue at play here. I have long found the meta-analyses of public health studies such as masking to be uniquely challenging. Meta-analysis seems better suited to randomized placebo-controlled trials comparing this drug to that. The uncertainties in public health studies seem too great for the bigger data of a meta-analysis to overcome.

For example, for masking trials, those randomized to wearing a mask or not wearing a mask cannot be blinded: A person in the unmasked arm may have snuck a mask on here and there or conversely, the masked group may have spent a day or two, or ten, unmasked. This introduces enormous and irreducible uncertainty regarding just what is happening.

Plus, focus on this particular meta-analysis—and it alone—is misguided. Other meta-analyses by experts found a “significant protective effect” from masks.

And perhaps most important of all, we have witnessed masks at work over the last three years. Yes, this is anathema to the purists who require evidence, not observation, but how else can we explain the unprecedented low (almost zero) rates of influenza in the 2020-2021 respiratory season other than by close adherence by the many to the protective measures taken to prevent disease transmission.

Despite all the current kerfuffle, I doubt anyone will change their mind about the issue. Maskers like me will continue to believe and the anti-maskers, though perhaps walking a little taller for a moment, will continue to blame masks for everything from childhood development delays to crashing economies to a dulling of the public sensitivity to the ever-encroaching State that will, when completed, supposedly deprive everyone of all freedoms.

More alarming than the mask debate is to note that this seems to be the latest in a disturbing trend of those with a political ax to grind revisiting the three-year history of the pandemic in a way that needlessly casts doubt or misdirects attention.

Already this year, we have heard that one US agency finds weak evidence that the viral strain that overwhelmed the world was released from a lab in China; we hear ever more about how vaccines are causing a global increase in deaths albeit from a discredited source and we see more focus on the profit motive as an explanation for how the pandemic played out.

Of course, we must look back to learn from our mistakes and make our response to the inevitable next pandemic more effective. Much of the current discussion though seems to involve different groups trying to gain the upper hand of a much larger political—not scientific—debate. And the only thing we can learn from this type of proxy war is this: Everyone who is focused on the politics or media hype of the issue, at the expense of the actual science behind the public health policies, should kindly shut up. Full stop.