Weekend Long Reads: The Vaccine Efficacy Studies

by Kevin Schofield

Over the past 18 months, I've read well over a hundred research papers on COVID-19, treatments, and vaccines. This week's "long read" is hands down the most informative of all of them.

One of the essential tenets of science is that it must be repeatable: Every time an event happens, we should expect the same result. In practice we often don't see precisely the same result, even in laboratory conditions, because of experimental error, contamination, sampling error, unknown confounding factors, and bias (intentional or otherwise). That's why in the world of science a single research study alone isn't enough to establish new knowledge; the scientific community waits until other researchers have replicated the study and independently confirmed the results.

In our mad rush to save lives by developing treatments and vaccines for COVID-19, we have often substituted a single carefully crafted and skeptically reviewed clinical trial for the greater assurance that we would get with multiple complete studies — at least for the purposes of "emergency use authorization" by the U.S. Food and Drug Administration (FDA). But with the passage of time, and the need to recertify the vaccines in multiple countries, there is now a substantial number of vaccine studies that have been published. Together they give us much higher confidence in our estimates of the effectiveness of the vaccines.

A group of researchers at Johns Hopkins University have written a "survey paper" that compiles the results of several dozen vaccine efficacy studies, across 13 different vaccines. Just tabulating them is interesting in its own right: They catalogued 51 individual studies of the Pfizer vaccine, 14 for the Moderna, 22 for AstraZeneca, and lower numbers for the others (the researchers excluded many studies because they didn't meet their quality criteria).

Comparing the studies highlights that there are many potential goals of a vaccine, including preventing infection entirely, preventing asymptomatic infection, preventing symptomatic disease, preventing severe disease, and preventing death. It's easy to overfocus on whether vaccines prevent infection entirely; while that is an extremely desirable outcome (because it stops further spread of the virus), the other potential outcomes also have enormous public health benefits. There are all sorts of harmless viruses that circulate quietly among the human population, and if vaccines can reduce the coronavirus to "harmless," we could go back to our normal lives.

The Johns Hopkins researchers begin with a side-by-side comparison of the 13 vaccines for their ability to prevent symptomatic disease based upon their Phase 3 clinical trials. The effectiveness ranges from 65% to about 95%. We sort of knew that some of the vaccines were better than others, but it's helpful to see the data presented together.

Then they cut the data a different way and show us the full set of studies for a given vaccine. It's particularly fascinating to look at the results for the Pfizer vaccine, because there are so many studies — it's a lot of data points. They break out the studies between those who received just one shot and those who got the full two doses, and one can clearly see the impact of that second shot. The researchers also color-code the studies based upon the predominant variant at the time and place the study was done. The Pfizer vaccine is clearly not as effective at preventing infection by the delta variant, but it's still extremely effective at preventing symptoms, severe disease, and death. Having multiple studies showing the same result is comforting.

The Moderna vaccine has considerably less data, but we can still see that the studies show it's a bit less effective against delta (though better than Pfizer). The Johnson and Johnson/Janssen vaccine, which only requires one dose, is substantially less effective, but that's been known since it was first approved; it's still highly effective at preventing serious disease and death, and for those who can't arrange for a two-dose vaccination, it could still save their life.

The researchers finish their paper by flipping the data around once again and presenting all the studies against a particular variant. The bad news is that all of the vaccines are less effective against the delta variant; the good news is that particularly against severe disease they are all still highly effective.

Science is winning: We now have repeated studies showing that the vaccines work. And especially for the Pfizer vaccine, the evidence is now overwhelming.

A systematic review of COVID-19 vaccine efficacy and effectiveness against SARS-CoV-2 infection and disease

Featured image is attributed to The Focal Project (under a Creative Commons, CC BY-NC 2.0 license).