Weekend Reads | COVID Responses Across the U.S.: What Worked, What Didn't, and What Made a Difference

by Kevin Schofield

If you're like me, then several times over the past three years, you've said to yourself, "Someday, the pandemic will be over, and we'll be able to look back and know what actually worked and what was just nonsense." While COVID isn't entirely behind us yet, most of the emergency declarations are expiring, and the public health community is now finally getting around to understanding who responded well, who didn't, and what made a difference.

This weekend's read is an important first piece of that puzzle: Assembled by a long list of researchers led by the University of Washington's Institute of Health Metrics and Evaluation, it's a comprehensive look at how each of the 50 U.S. states did in limiting COVID infections and deaths, with analysis on how those results correlate to the specific steps each state took (or didn't) to limit the impact of the virus — as well as the pre-existing public health, economic, and political situation in each state that may have set it up for success or failure. Finally, it looks at whether each state's COVID response resulted in specific economic and/or educational trade-offs, for example, whether student achievement suffered more in states that took more aggressive steps to control the virus.

One of the challenges in doing a study like this one is in making it an "apples to apples" comparison, since our 50 states didn't all start with the same pre-existing conditions. It's now known that older people and those with certain health conditions tended to have more severe cases of COVID if they caught the disease; that would mean that states with older and less healthy populations would look worse in comparison with younger and healthier populations regardless of how they specifically responded to the coronavirus. So the researchers "normalized" the infection and mortality statistics for each state to account for age and "co-morbidities" — two factors that are known to have direct biological connections to COVID severity — but not of a range of other socioeconomic factors such as race/ethnicity, poverty level, and income inequality that they wanted to measure separately.

The researchers' report is a wealth of data on what made a difference, for better or for worse, as well as what turned out not to be consequential. While it likely won't be the final word, it will inspire years of deep dives into what happened during the pandemic.

For example, states with a larger population of Asian and Pacific Islanders (API) had lower infection rates and fewer COVID-caused deaths, while states with higher Black and Hispanic populations had higher COVID death rates — but not necessarily higher infection rates. That's a phenomenon that repeatedly appears in the data: In some cases infection rates and death rates are tightly related, but in others they are independent. Put another way: How to stop the spread of a virus and how to stop it from killing people are two different questions with only somewhat related answers.

To that end, states that instituted some set of mandates (such as closing schools, wearing masks in certain locations, "stay-at-home" orders, or vaccine mandates for public employees) had lower COVID infection rates than those that didn't, but not consistently lower COVID death rates. Further, few of the specific types of mandates individually made a measurable, consistent difference. Among the ones that did, vaccine mandates for state employees lowered both infections and deaths, and vaccine mandates for school employees lowered deaths but not infections. For the others the results varied widely from state to state, probably related to whether they were enacted in combination with other mandates and policies that reinforced each other.

The researchers also looked at the behavioral response, particularly how much of the population in each state got vaccinated and how many people regularly wore masks. As might be expected, higher vaccine coverage corresponded to a big decrease in both infections and deaths. Mask use, on the other hand, tended to decrease infections but not deaths — another example of an action that affects one but not the other.

The report spends a fair amount of time looking at the "pre-existing conditions" in each state, which is important because it tells us what we should be doing now to decrease the impact of the next pandemic. High-poverty states tended to have both higher infection rate and more deaths; those with higher income inequality and more people without health insurance saw higher death rates but not necessarily higher infection rates, which makes sense since more people would be unable to seek the kind of care that would help them to survive COVID. On the flip side, a few factors lowered both infection and death rates, most notably more years of education and better access to and quality of health care. A few factors additionally helped to lower infection rates (but not deaths), including state-funded family or sick leave, and greater trust in the scientific community.

One factor jumps out as particularly interesting: States where there is a higher level of "interpersonal trust" had lower infection and COVID mortality rates. That's a challenging one, because it goes beyond traditional policy actions by officials and speaks to a larger social phenomenon of community bonds breaking down, perhaps making it more difficult for those communities to band together and support each other in taking difficult steps during a time of crisis.

The researchers also looked at a number of political factors to see which made a difference, and the results were not as straightforward or predictable as you may expect. Surprisingly given the political climate and pressures, overall the states with a Republican governor (the official primarily responsible for imposing mandates in response to COVID) did not fare worse than states with a Democratic governor. On the other hand, the states where a majority voted for Trump in 2020 had both higher infection rates and higher mortality rates. Also, among "red" states there was higher vaccine uptake in those where the average education level was higher, suggesting that while partisanship was a big factor it wasn't the only factor and may not have been the most important one. That said, the researchers remind us that trust is partisan: Studies have shown that people express less trust in government when the sitting president is not from their party.

There were also some surprising results regarding the economic and educational effects of the COVID pandemic, states' responses, and infection and mortality rates. The researchers found that states with stronger economies before the pandemic did not necessarily have lower infection and death rates, and that none of the mandates had a statistically significant effect on a state's GDP; nor did mask use, vaccine coverage, or reduced mobility. Also, employment levels coming out of the pandemic only correlated with restaurant closure mandates and not any other type of mandate — though for unexplained reasons, higher mask usage did correlate with lower employment levels.

If you've been following the news, you're probably aware that across the United States, student achievement dropped dramatically during the pandemic. But the researchers found that the drop in achievement wasn't connected to school closures, stay-at-home orders, or gathering restrictions. It did, however, correlate with mask and vaccine mandates, and those states that imposed any package of mandates at all. The achievement drop also correlated with mask use and overall vaccine uptake. The researchers pondered this data and couldn't come up with a good explanation, but suggested that parents who bought into masks and vaccines may also have been more willing to self-impose keeping their kids out of school longer in favor of schooling them at home or remote participation in classes.

The researchers end their report with a discussion of some of the policy implications of their study — though we will certainly be debating the lessons of the COVID pandemic for years. Certainly there are some "pre-existing conditions" such as poverty, income inequality, limited access to health care and health insurance, and trust issues that made a difference in some states over the past three years and that we should be working to improve now (arguably regardless of the pandemic, we need to work on them). They also suggest some ways to deal with the "trust gaps" next time: early, targeted community-based efforts that use local, trusted spokespeople who will appeal to those who doubt the government as well as clear, transparent, and timely communication to build public trust. There is also work to do to understand the educational trade-offs made, and to provide more support to help the lowest-achieving students catch up.

The researchers begin their report by quoting a columnist for The Atlantic in saying that COVID "defeated America" and "humbled and humiliated the planet's most powerful nation." There is much truth to that. Nevertheless, the results in this report gives us hope that we can do much better next time.

Assessing COVID-19 pandemic policies and behaviours and their economic and educational trade-offs across US states from Jan 1, 2020, to July 31, 2022: an observational analysis

Featured Image: There is still a lot to be learned from each state's COVID-response. This new study considered many factors when evaluating every state's effectiveness at preventing transmission and deaths due to COVID-19. (Photo via Kits Pix/Shutterstock.com)