The current CDC guidance on masking in schools says that all children above the age of two should wear that at all times regardless of vaccination status. Yesterday, Atlantic contributor David Zweig pointed out that the CDC’s approach is quite a bit more stringent than what is being done elsewhere in the world:

The World Health Organization, for example, does not recommend masks for children under age 6. The European Centre for Disease Prevention and Control recommends against the use of masks for any children in primary school.

One of the main reasons for the strict CDC approach is a study out of Arizona which Director Rochelle Walensky began touting in September. The striking claim based on this one study is that schools without mask mandates “were 3.5 times more likely” to have a COVID outbreak than those requiring masks.

New @CDCMMWR data reinforce the benefits of masks and vaccinations in preventing #COVID19 outbreaks in schools.

Together, we can do our part to keep students and staff safe AND keep our schools open. https://t.co/AJ8kKyhYXG https://t.co/k4ljDBw4AG

— Rochelle Walensky, MD, MPH (@CDCDirector) September 28, 2021

Walensky touted this research on television and during a White House briefing and it was widely covered by the media. The study is a comparison of schools in two Arizona counties. Of the 210 schools with mask mandates only 16 had outbreaks while among the 480 schools without mask mandates there were 113 outbreaks. That’s where the 3.5 figure comes from. But according to eight experts consulted by Zweig, the Arizona study isn’t reliable:

The Arizona study’s lead authors stand by their work, and so does the CDC. But the critics were forthright in their harsh assessments. Noah Haber, an interdisciplinary scientist and a co-author of a systematic review of COVID-19 mitigation policies, called the research “so unreliable that it probably should not have been entered into the public discourse.”…

The study’s methodology and data set appear to have significant flaws. The trouble begins with the opening lines of the paper, where the authors say they evaluated the association between school mask policies and school-associated COVID-19 outbreaks “during July 15–August 31, 2021.” After reviewing school calendars and speaking with several school administrators in Maricopa and Pima Counties, I found that only a small proportion of the schools in the study were open at any point during July. Some didn’t begin class until August 10; others were open from July 19 or July 21. That means students in the latter group of schools had twice as much time—six weeks instead of three weeks—in which to develop a COVID outbreak.

When I brought this issue to Megan Jehn, the study’s corresponding author and an epidemiologist at Arizona State University, she acknowledged that exposure times varied across schools. The ones without mask mandates were open longer overall, she told me—but the difference was too small to matter. Their median start date was August 3, versus August 5 for the schools that did have mask mandates. In a follow-up correspondence, Jehn and McCullough wrote, “It is highly improbable that this difference alone could explain the strong association observed between mask policies and school outbreaks.”

Yet Ketcham said that a comparison of median start dates is insufficient. “If schools with mask mandates had fewer school days during the study,” he told me, “that alone could explain the difference in outbreaks.”

That’s just one problem with the study. An economics professor at Yale who carried out a large study of masking in Bangladesh called the Arizona study “ridiculous” because it didn’t control for the vaccination status of students and teachers. If the no making group had lower vaccination rates to start with, that would be important to know.

The researchers also made some mistakes with the precise schools included in their study. Zweig got a list of 891 public, non-charter schools in Maricopa county and found that “40 of them were virtual learning academies, about 20 were preschools, and about 90 were vocational programs associated with otherwise-listed schools.” When you subtract all of those from the total you wind up with 740 schools that were eligible for the study. But according to the research itself, they included 782 schools in Maricopa county.

Zweig repeatedly asked the authors for a complete list of the schools they used in the study and they refused to provide one until a few days before publication. At that point he discovered their list included a virtual school, a preschool and “80 entries for vocational programs that are not actual schools.” The authors admitted the inclusion of the virtual school was a mistake but blamed the other problems on the Arizona DOE.

Zweig argues that the size of the difference found in the Arizona research alone should have been a clue that something was off. The aforementioned Yale economist, Jason Abaluck, conducted a randomized study of 340,000 people in Bangladesh which found wearing surgical masks reduced COVID spread about 11 percent while using cotton masks had an impact that ranged somewhere between that of surgical masks and no impact at all.

It’s worth noting that the Bangladesh study didn’t have complete masking even in the best of cases. The use of masks was always below 50%. So it’s reasonable to imagine the effect of masking could have been larger in a controlled environment like a school where kids can be forced to wear masks while in class. Still, there’s a huge gap between the 11% difference found in the Bangladesh study and the 3.5 times impact found in the Arizona study.

Another study that did look at school students in Georgia found outbreaks were 21% lower in schools where kids wore masks. Curiously, the same study found schools where only adults were masked were 37% less likely to have an outbreak. If you assume that’s a realistic range (it’s double or triple the effect seen in the Bangladesh study) it’s still a far less dramatic difference than what the Arizona study found.

There’s more in the Atlantic piece but the bottom line is that the Arizona study, which at least partly accounts for our unusually strict approach to school masking among young kids, looks like an outlier. Given the many potential problems with it, the CDC shouldn’t have put so much emphasis on it.