The vaccine candidate sponsored by the University of Oxford and AstraZeneca has shown promise in preventing COVID-19 in Phase III clinical trials, but the evidence is not yet convincing enough to put it in the same category as competitors from Pfizer-BioNTech and Moderna. Consequently, it is likely that the Food and Drug Administration (FDA) will insist on more and better data before granting regulatory approval.
The preliminary results from the Oxford-AstraZeneca Phase III trial provided cause for optimism, but left many questions unanswered. While reporting an overall vaccine efficacy rate of 70 percent—a rate that exceeds the minimum 50 percent threshold set by the World Health Organization (WHO) and FDA for approving COVID-19 vaccines—the trial sponsors further announced that the efficacy of their vaccine was 90 percent for a limited subset of participants who received a half dose and then a full dose one month apart.
Just under 1 in 4 participants in the vaccine arm of the trial received the half-dose/full-dose regimen, which was the result of a manufacturing error and not a part of the original trial protocol. The rest, 3 out 4 trial volunteers in the vaccine arm, received two full doses one month apart.
For participants who received the full-dose/full-dose regimen, the observed vaccine efficacy rate was 62 percent, or 28 percentage points below the half-dose/full-dose regimen. Because so many more participants received two full doses rather than a half dose followed by a full dose, the average efficacy rate when combined across the two dosing regimens was closer to the efficacy rate found among those receiving the full-dose/full-dose regimen.
On the one hand, the reported 90 percent efficacy is potentially very positive news, especially given the vaccine’s lower price and less demanding handling requirements compared to the alternative candidates sponsored by Pfizer-BioNTech and Moderna. However, the trial sponsor has not yet explained why the half-dose/full-dose regimen would be so much more effective than two full doses, and many vaccine experts have been puzzled by this finding. Further, there appears to be some disagreement among the officials involved in the trial over whether the administration of a half-dose/full dose regimen to some participants was truly a mistake or an avenue they had purposefully pursued. Most importantly, it cannot be ruled out that the difference in the observed efficacy across the two dosage regimens is a consequence of small sample sizes and statistical chance rather than a meaningful clinical reaction based on the dosing adjustment.
Statistical modeling of vaccine efficacy trials reveals that, while there is indeed a high likelihood that the Oxford-AstraZeneca vaccine is more efficacious when dosed properly, the reported differences do not reach the standards of statistical significance that are used in most scientific studies.
For instance, in the WHO and FDA guidance, the agencies specify that vaccine candidates need to demonstrate efficacy of at least 50 percent in Phase III trials — and have sample sizes sufficient to ensure there is no more than a 2.5 percent chance that the true efficacy of the vaccine is less than 30 percent. The Oxford-AstraZeneca vaccine is likely to satisfy these requirements, but it also must be compared to the efficacy rates of the Pfizer-BioNTech and Moderna vaccines, for which there is high certainty of efficacy of nearly 90 percent. The sponsors of the Oxford-AstraZeneca candidate claim their vaccine could produce comparable results, but the data revealed so far leaves rooms for doubt.
Based on the case counts in the Oxford-AstraZeneca vaccine trial, efficacy modeling shows there is a 6.3 percent—or around 1 in 16—chance that the two dosage regimens used in the trial would have an observed efficacy gap of at least 28 percentage points when their true efficacy was the same. This is below the level of statistical confidence normally required in the vaccine approval process.
We can reach this conclusion based strictly on data released by the sponsors themselves. They have disclosed that 131 trial participants contracted COVID-19 and that the overall efficacy rate was 70 percent, implying that 30 of the positive cases emerged in the vaccinated arms compared with 101 in the placebo arms. Out of more than 23,000 total participants, 8,895 trial participants received two full doses of the vaccine, while 2,741 received a half dose followed by a full dose.
Using this data, a mathematical model can simulate a range of results assuming two trials are run separately—one for each dosing regimen—while holding constant their combined results to match the publicly reported results from the sponsors. That is, the model randomly allocates the observed positive COVID-19 cases among the two placebo and two treatment arms in proportion with the number of trial participants. Then, it calculates how often the difference in the observed efficacy rates across the two trials is greater than or equal to 28 percentage points in 100,000 simulated sets of trials.
This modeling reveals that, while that difference is large, it would occur by chance 6.3 percent of the time even when there is no actual difference in the true efficacy of the vaccine based on the dosing changes. For every 1,000 times the simulation is run, a difference in vaccine efficacy rates that is 28 percentage points or more is observed approximately 63 times. Although promising, these results imply that the difference in the observed outcomes could be the result of luck, rather than true differences in efficacy driven by an adjustment in dosage.
Especially when combined with other concerns about the generalizability of the group of individuals who received the smaller initial dose—Oxford-AstraZeneca has revealed that no participants older than 55 received it—the results call for patience as more data is collected and experts develop a better scientific understanding of why individual immune response could differ depending on the size of the initial dose. For these reasons, AstraZeneca is unlikely to file an application for an emergency use authorization in the U.S. until more data become available, which will likely be in January. The company also is planning to launch a second trial that focuses exclusively on the half-dose/full-dose regimen.
Whereas the Pfizer-BioNTech and Moderna vaccines appear to be highly effective in combating COVID-19, it is too soon to tell whether the Oxford-AstraZeneca vaccine measures up.
Kieran Allsop is a research assistant at the American Enterprise Institute. James C. Capretta is a resident fellow and holds the Milton Friedman chair at the American Enterprise Institute. Scott Ganz is a research fellow in economic policy studies at AEI and an assistant professor at Georgia Tech’s School of Public Policy. Capretta and Ganz are the authors of “Awaiting the Signal: Assessing the Efficacy of COVID-19 Vaccines,” published in October by AEI.
Please note that we at The Dispatch hold ourselves, our work, and our commenters to a higher standard than other places on the internet. We welcome comments that foster genuine debate or discussion—including comments critical of us or our work—but responses that include ad hominem attacks on fellow Dispatch members or are intended to stoke fear and anger may be moderated.
With your membership, you only have the ability to comment on The Morning Dispatch articles. Consider upgrading to join the conversation everywhere.