Fact check: Post misleads on effectiveness of J&J COVID-19 vaccine in older adults
Corrections & Clarifications: This story was updated June 22 with a reworded claim that provides a more nuanced summary of the point made in the Facebook post being fact-checked here. Other references were updated for clarity. This does not change our rating for this post, which remains False. This fact-check is based on the original version of a Facebook post, which was updated after this check published to more precisely reference some of the data involved.
The claim: Data shows J&J COVID-19 vaccine may not be effective in older adults
Johnson & Johnson's COVID-19 vaccine made headlines in early April after six women developed extremely rare blood clots following vaccination.
Use of the one-dose shot was paused for several weeks but resumed April 23 after the U.S. Food and Drug Administration and the U.S. Centers for Disease Control and Prevention found the vaccine was overall safe and effective.
Despite that, doubts about the J&J vaccine, particularly in older adults, linger on social media.
"The clinical trial indicates that in subjects 65 years and older, the (J&J) vaccine may be only 39% effective, and in subjects 75 years or older, the age group that comprises about 60% of all COVID-19 deaths, the vaccine may be 0% effective," claims a Facebook post May 18 by California-based Physicians for Informed Consent, or PIC. PIC describes its goal as enabling doctors and the public "to evaluate the data on infectious diseases and vaccines objectively, and voluntarily engage in informed decision-making about vaccination."
The post, which includes a chart detailing these and other alleged statistics, claims the vaccine "may not be effective" in people 60 and older with severe risk factors for COVID-19.
Fact check: Peer-reviewed studies have shown safety, efficacy of COVID-19 vaccines
PIC general counsel Greg Glaser told USA TODAY the numbers in the post and its accompanying table were largely based on the FDA's emergency authorization memorandum for J&J's vaccine and PIC's own compiled references.
The numbers PIC put forth overstate or oversimplify data in the FDA's official findings.
Vaccine efficacy versus vaccine effectiveness
Though the question of how well a vaccine works is often discussed as vaccine "effectiveness," the actual term is vaccine efficacy, a slightly different but related measure. PIC used the efficacy term in its chart, but referred to "effectiveness" throughout the text accompanying the post.
Efficacy is a value obtained from randomized clinical trials and represents a best estimate of how well the product being tested (a vaccine in this case) works in an idealized condition, said Natalie Dean, assistant professor of biostatistics at the University of Florida.
In the setting of a vaccine trial, this is determined by comparing the number of disease or infection cases occurring among the vaccinated group with that in the unvaccinated, or placebo, group, according to the CDC.
If vaccine efficacy is 0%, this means the vaccine doesn't considerably reduce the risk of a disease or infection for the group who received it compared with those who received a placebo. A vaccine efficacy nearing or at 100% means it most likely does.
Effectiveness, on the other hand, gives an idea of how well a drug or other treatment works outside a clinical trial under real-world settings, though it relies on a similar calculation.
Fact check: No, the CDC did not release data showing 7 in 10 Americans decline COVID-19 vaccine
Dean said it's likely vaccine effectiveness will match vaccine efficacy, but there are a variety of reasons why they might not: age of the vaccine recipient, presence of underlying chronic illnesses and use of medications, as well as how a vaccine is stored and administered or if the targeted virus mutates.
39% is from a confidence interval
Glaser said PIC drew its figures from Table 12 of the FDA's EUA memorandum. The table presents vaccine efficacy, or VE, as percentages observed at 14 and 28 days after vaccination.
The values listed for those 65 and older show J&J's COVID-19 vaccine is quite efficacious – 76.5% after 14 days and 68.6% after 28 days. That's better than the numbers for the 18-to-64 age group, which were 64.7% after 14 days and 65.1% after 28.
Because these numbers are based on a sampling of the general population – about 40,000 participants across the USA, South Africa and Latin America – 76.5% and 68.8% represent only estimates of vaccine efficacy, specifically what is called point estimate, said Dr. Angela Branche, co-director of the Vaccine and Treatment Evaluation Unit at the University of Rochester Medical Center.
Point estimates are typically accompanied by a range of values called a confidence interval, which is where PIC gets its "39%" effectiveness. The 95% confidence interval reported by J&J means there is a 95% likelihood the actual value is 39% to 85%. PIC references only the low end of that range and ignores the actual estimate value of 68.6%.
"What (PIC) is reporting here is extremely misleading because you cannot interpret confidence intervals as efficacy. Efficacy is what it is – it's a point estimate," Branche told USA TODAY.
Confidence intervals only express the precision scientists are able to assign to that number but aren't used to represent vaccine efficacy, said Dr. Matthew Laurens, an infectious disease specialist and vaccine researcher at the University of Maryland School of Medicine.
Laurens said the range of the confidence interval is a product of how many people are enrolled in the study.
"If you look at the confidence interval for the 18 to 64, it's much tighter because the numbers (of participants within that age group) are much higher. The more people we include, the more precision we have with the vaccine efficacy," he said.
Despite there being not as many older participants as younger ones, Laurens said what was especially telling about J&J's vaccine is that it performed well against COVID-19 variants.
"Of the three vaccines that have received emergency authorization in the United States, (J&J's vaccine) is the only one that has been tested in clinical trials in South Africa (and Brazil)," he said. "Those variants were tested by this vaccine, still protected and still showed efficacy no matter the region whether it was North America, southern Africa or Latin America."
Fact check: Fauci didn't say half of CDC, FDA employees refused COVID-19 vaccine
This consistency across geographies, age and race is far more crucial and spoke to the fact J&J's vaccine efficacy resided in the upper 60% range than the lower bounds of a confidence interval, Laurens emphasized.
Vaccine efficacious for 75 years and older
PIC's claim that J&J's COVID-19 vaccine "may be 0% effective" for people 75 and older also mishandles the FDA data.
Table 12 shows that at the 14-day endpoint, VE is listed pretty high at 89.7%. VE at the 28-day endpoint is left blank on the chart.
PIC refers to this as indicating it may be 0% effective, but the opposite is true, Branche said.
"Efficacy only goes up as you move forward in time," she said. "You're least likely to be protected early on in that 14-day period. As you move off toward 28 days and so forth, then you really have your best protection at that point."
The VE was probably left blank at the 28-day point because J&J's trial didn't have enough time to collect all the data as the FDA's emergency use authorization was underway, Branche said.
The efficacy for that period could only be higher than the 14-day level, which is estimated at 89.7%. It couldn't be as low as 0%, as the post claims.
Data does show J&J's vaccine works for older adults with COVID-19 risk factors
A lower VE estimate was observed for those 60 and older with preexisting conditions, particularly at the 28-day endpoint (65% versus 42%). A reason for this could be "imprecision owing to fewer cases and shorter follow-up in this subgroup," a spokesperson for Janssen, a pharmaceutical company owned by J&J, said in an email to USA TODAY.
"Furthermore, Kaplan-Meier curves (a statistical tool for estimating when there's incomplete data) indicated that the cumulative incidence of cases among vaccine recipients 60 years of age or older with coexisting conditions was similar to that in the overall trial population, which suggests a similar vaccine efficacy," the company said.
The FDA's EUA memorandum supported this point. It said no COVID-19 deaths or cases requiring medical intervention took place 28 days or more after vaccination among that high-risk subgroup.
Our rating: False
Based on our research, we rate FALSE the claim that data shows the J&J COVID-19 vaccine may not be effective in older adults. The statistics provided by Physicians for Informed Consent in its post and accompanying table were taken from the FDA's emergency use authorization for J&J's vaccine but are misrepresented.
The 39% figure is the lower bound of a confidence interval for vaccine efficacy 28 days post-vaccination in the 65 and older age group. It doesn't represent the actual efficacy. Confidence intervals are meant to give information about the precision of a value such as vaccine efficacy, not to represent vaccine efficacy itself. The claim the vaccine could be "0% effective" among those 75 and older is wrong. Experts say the 28-day figure can't be lower than the 14-day figure, which the report estimates at 89.7%. Although vaccine efficacy was lower for those 60 and older with preexisting conditions, that's because there were fewer COVID-19 cases and shorter follow-ups in this group. No COVID-19-related deaths or cases requiring medical intervention were observed in these individuals, according to the FDA.
Our fact check sources:
USA TODAY, April 14, What do I do if I've gotten the Johnson & Johnson COVID-19 shot? Your questions answered
USA TODAY, April 25, Pause on Johnson & Johnson COVID vaccine in US lifted by FDA, use to resume
U.S. Centers for Disease Control and Prevention, accessed May 28, COVID-19 Vaccinations in the United States
Science-Based Medicine, Feb. 18, 2019, The strange sage of Peter Gøtzsche and Physicians for Informed Consent
Natalie Dean, June 4, Phone interview with USA TODAY
U.S. Centers for Disease Control and Prevention, May 18, 2012, Section 6: Measures of Public Health Impact
The New York Times, March 3, What Do Vaccine Efficacy Numbers Actually Mean?
Gavi, The Vaccine Alliance, Nov. 18, 2020, What is the difference between efficacy and effectiveness?
U.S. Food and Drug Administration, Feb. 4, Janssen COVID-19 Vaccine EUA FDA Review Memorandum
Dr. Angela Branche, June 4, Phone interview with USA TODAY
Dr. Matthew Laurens, June 4, Phone interview with USA TODAY
Janssen, June 1, Email interview with USA TODAY
Dr. Paul Offit, May 27, Phone interview with USA TODAY
Thank you for supporting our journalism. You can subscribe to our print edition, ad-free app or electronic newspaper replica here.
Our fact check work is supported in part by a grant from Facebook.
This article originally appeared on USA TODAY: Fact check: False claim about effectiveness of J&J COVID-19 vaccine