Don't trust the polls: Pollsters are increasingly unable to predict American thought

A remarkable opinion piece appeared recently in the Star-Ledger, New Jersey’s most prominent newspaper. It was written by Patrick Murray, the director of the Monmouth College poll that has been widely cited in national news media for years. In his opinion piece, Murray concedes that his polling on the New Jersey gubernatorial election was a failure.

The Monmouth poll consistently forecasted an easy victory for incumbent New Jersey Gov. Phil Murphy. But election night found Murphy running neck-and-neck with challenger Jack Ciattarelli and possibly even being defeated by him.

Ultimately, Murphy won by about two points, but the polling narrative of an easy win for the Democrat was not just a bad call, but a message to pollsters everywhere that their surveys, like so much else in American politics, have been contaminated by political polarization.

The pitfalls of poll predictions

One of the things pollsters have known for years but have been unwilling to face up to, because their business is so lucrative and marketable, is that a combination of technological changes, like modeling, and a disposition by those being polled to lie or deceive pollsters with answers that they deem to be more socially acceptable have rendered these surveys practically useless, especially in close elections.

The excuses trotted out by pollsters after the election in 2016, in which Hillary Clinton was predicted to be an easy winner, were a combination of two lame explanations: The first is that the polls were correct as far as the national electorate was concerned – Clinton did win the popular vote – but that state-level polls were inaccurate. The second was that missing the level of Trump’s support was the result of respondents not being upfront with pollsters.

This was known as the “shy Trump voter” problem: Many voters thought that admitting to supporting Trump was not entirely respectable, so the respondent told the pollsters what they thought the pollsters wanted to hear – which was a preference for Clinton.

Poll respondents lying to surveyors is no new thing. Back in 1982 Los Angeles Mayor Tom Bradley, an African American, lost his race to be California's governor, despite being shown consistently ahead in the polls.

If that were not enough proof to pollsters that people would lie to them in order to convince their questioners that they are not racist, the gubernatorial camping of Douglas Wilder in Virginia in 1989 also showed the African American candidate with a solid lead. Yet, he only won by a mere half of 1%.

Paranoia with the pollsters?

Those contacted by phone seem to believe that they are being judged by the caller. Rather than risking to appear racist for not professing support for a Black candidate, or feeling “shy” about admitting to be a Trump supporter, they don’t admit their real preference. Even worse, many people on the right may even refuse to be polled by a representative of the "lame stream media."

High refusal rates force pollsters to use statistical models to simulate the views of the nonresponders. They have also come to depend too heavily on those whom they calculate to be "likely voters" and don’t account for those occasional voters who will come out to use their vote as a gesture of protest. Pollsters basically have no idea who is actually going to show up on Election Day.

Voters should also understand that there is a wide variation in the quality of polls.

Glenn Harlan Reynolds: Election results: Terry McAuliffe, Virginia Democrats triggered their own defeat

The worst are the polls that the campaigns themselves conduct. When they release results, the polls usually inflate the support for the campaign that has commissioned the poll.

News organizations such as The New York Times and The Washington Post partner up with university-based polls to give them some intellectual heft, but it is unlikely that a supporter of Trump would be disposed to answer the questions of pollsters identifying themselves with two liberal media outlets collaborating with academic institutions.

Ross K. Baker: Democrats need an autopsy to figure out why 74 million Americans voted against them

I know Patrick Murray. He was trained at my university, and his candor reflects well on his character. But I think he should have gone even further and issued a broader indictment of the commercial polling industry that has shown its flaws as far back as the disastrous Literary Digest poll of the 1936 election, which showed Kansas Gov. Alf Landon beating Franklin D. Roosevelt because its poll tallied only the responses of those who owned phones in that year in the depth of the Great Depression.

As fewer people respond to pollsters and more react hostilely or give ingratiating answers, the polls become less believable and less valuable in understanding what Americans are thinking.

Ross K. Baker is a distinguished professor of political science at Rutgers University and a member of USA TODAY's Board of Contributors. Follow him on Twitter: @Rosbake1

You can read diverse opinions from our Board of Contributors and other writers on the Opinion front page, on Twitter @usatodayopinion and in our daily Opinion newsletter. To respond to a column, submit a comment to letters@usatoday.com.

This article originally appeared on USA TODAY: Why are polls so wrong? They are poor predictors of election outcomes