What the New Poll Favoring Trump Got Wrong and the Pundits Missed

Former US president Donald Trump playing golf at Trump International Golf Links & Hotel in Doonbeg, Co. Clare, during his visit to Ireland. Picture date: Thursday May 4, 2023. Credit - Brian Lawless-PA Images

No one was more suspicious of the widely showcased newest ABC-Washington Post poll than ABC anchor George Stephanopoulos who questioned its accuracy live on air on its Sunday May 7 release. Stephanopoulos was suspicious given the inconsistencies in the survey which both purported to overwhelmingly condemn Trump’s likely illegal conduct supposedly showing Trump leading Biden in a prospective 2024 presidential election matchup.

Stephanopoulos was right to be suspicious of ABC’s co-sponsored survey as this poll was apparently designed for media drama but not an accurate public opinion metric. For just this reason, Robert Kaiser, the longtime managing editor of The Washington Post, this poll’s other co-sponsor told us he refused to showcase such polls in an earlier era calling it “manufactured news.”

Of course, political professional point out, polls done this far out have little value, and much can and will change in eighteen months by November 2024. In fact, Ronald Reagan, Bill Clinton, and Barack Obama are just three recent presidents who rebounded from comparable sagging public polls to ultimately win 18 months later. We acknowledge the polling around 40 percent favorability is nothing for the Biden Administration to gloat over, however, according to the Gallup Poll, it is consistent with almost all the seven prior presidents at this time except for President H.W. Bush following the Iraqi invasion of Kuwait and President George W. Bush following the 9/11 attack on the U.S. In fact, George H.W. Bush enjoyed a 73% favorability in September 1990 but then went on to lose his reelection to Bill Clinton.

In fact, there are piles of contrasting polls. There are some polls where Trump beats Biden – such as the right-leaning Rasmussen polls and Stagwell’s Harris Poll. Trump has consistently dominated across both polls dating back years. But most major opinion polls, on average, Biden leads Trump by 2 percent, including recent polls done by Yahoo News, The Wall Street Journal, Quinnipiac University, and Morning Consult among many others. Even Rupert Murdoch’s right-leaning New York Post grudgingly concedes that “Biden beats Trump big” across early polling.

Despite the borrowed ABC/Washington Post branding, the consultant who conducted this research makes for a case study in shoddy methodological practices.

First, the ABC-WaPo sample size of 1,006 adults surveyed is jarringly small, compared to the sample sizes of peers such as 6,000 at Morning Consult. Even other Trump-favoring polls such as the Harris Poll had a sample nearly twice as large, with 1,850 respondents, increasing accuracy and reliability. The Edelman Trust Barometer samples up to 35,000 respondents.

Even other pollsters have attacked the methods of this Abt Associates poll for ABC/Washington Post pointing out that it strangely included all adults in its sample of 1,006, rather than just registered voters, and that breaking out registered voters brings the sample down to an even smaller 900. Furthermore, analysis of subgroups like young people, independents, Hispanics and Black Americans are so tiny they are not valid. Pollster Cornel Belcher commented “The poll really is trash, and I don’t say that lightly because I’ve had respect for their polling in the past.”

Second, an amazing 25% of respondents in the flawed ABC-WaPo poll were reached by landline in lieu of cell phones. It has already been empirically shown, repeatedly, that landline respondents tend to skew Republican dramatically. Indeed, it is hard to find any other poll with such a high proportion of landline responses.

This was akin to the infamous old Roper Poll error which proudly but wrongly predicted for the Chicago Tribune Dewey beating Truman the worst call in political history. Amazingly, the defensive chairman of the Roper Poll still did not concede defeat claiming ““Clearly they were wrong in determining the election. I think the 1948 polls were more accurate than the 1948 election.” Surely a man ahead of his times – even if wrong!

By contrast, only 13% of the Quinnipiac Poll responses were landline, while cutting-edge firms such as Stagwell and Morning Consult have transitioned to online public opinion polling in lieu of phone calls. The WaPo-ABC poll also disclosed that of the “752 interviews completed via cell phone, 583 interviews were with adults in cellphone-only households.” Thus the 169 “cell” interviews also favors landline respondents which were especially inclined to pick up the phone on a cold call. Unlike with online polls there was no adjustment for phone call respondents’ propensity to respond.

Third, the pollsters even fudged the numbers in ways large and small. The pollsters supposedly “weighted to correct for differential probabilities of selection among individuals who are landline-only, cellphone-only, or dual users”, but this impenetrable technical jargon is actually nonsense. In simple English, the pollsters have baked in all kinds of unknown, unverifiable and likely inaccurate assumptions into their model – as they wouldn’t have ended up with 25% landline respondents with a more accurate model, or the fact that they had 13% more self-identified GOP respondents than Democratic respondents.

Likewise, unlike even other more careful Trump-favoring surveys such as the Harris Poll; the ABC-WaPo poll did not weigh the results to match the demographic makeup of the population along crucial vectors such as marital status, income, employment, household size, and political party, meaning the sample of 1,006 respondents could be massively unrepresentative along these dimensions. The only demographic factors considered by the WaPo-ABC poll were the bare minimum – sex, region, age, education, and race.

Given all these methodological flaws, it is hardly surprising that even the self-disclosed “sampling error,” or margin of error, of the WaPo-ABC poll is significantly higher than that of peers at +/- 3.5%. This is 40% higher than the sampling error of the Quinnipiac Poll, 2.5%, and three times that of Morning Consult, 1%. If this were a rigorous quantitative exercise, with such a high margin of error, one would not be able to conclude that Trump is leading Biden at all, or vice versa, making this entire study basically useless.

Simplistic media headlines trumpeting its findings that Trump supposedly beat Biden gloss over the dubious numerical hogwash of the poll’s administrator, Abt Associates. Perhaps they even intentionally wanted to subvert best practices and generate provocative polling results to gain the prominence of stature and the expert academic authority they lack. After all, provocative, contrarian polls can help outlets compete commercially in the attention economy.

Considering even the most reputable pollsters got it wrong in 2018, 2020, and 2022, it is unfortunate that too many pollsters seek to obfuscate their underlying models through an artificial haze of impenetrable mathematical jargon. Yes, where was that predicted Red Wave to sweep the nation this past fall?

It is also past time for the political pundits to start asking more questions about shady polling practices and statistical sophistry rather than merely trumpeting inflammatory horse race headlines. Not all polls are created equal. There should be no shame in challenging double talk without being intimidated by technical jargon, and having confidence that you are not the village idiot when asking a question about polls of questionable quality.

I have no doubt that 92-year-old Abt Associates founder Clark Abt would shudder at such specious pseudo-science under his name if he examined this work. The first author of this article knew him decades ago as a pioneering MIT trained mathematical political game theorist and brilliant educational consultant.

Abt’s former Cambridge neighbor the Harvard economist John Kenneth Galbraith cautioned to one of us in 1974, “There are two kinds of forecasters: Those who don’t know and those who don’t know they don’t know.”