Marquette Law is releasing its 75th Wisconsin survey today. Here's what's new about this poll
Now in its 12th year, the Marquette Law School Poll is releasing its 75th Wisconsin survey today.
But like other pollsters, Marquette’s Charles Franklin is making major changes to the way he conducts his survey amid the growing obstacles facing traditional polling.
Because the poll is widely reported on, closely read in the political world, and has a tradition of transparency, I thought it would be helpful to talk through these changes — and what is driving them — with Franklin, the poll’s director and a colleague of mine at Marquette's Lubar Center.
Here is a transcript of our conversation, edited for clarity and length:
Why are pollsters changing the way they measure public opinion?
Franklin: Primarily because it’s become so hard to get people to answer the telephone … (and) the people that do answer are less reliably like the population we’re trying to sample. And so, pollsters generally and we in particular have moved to new methods that make it more reliable to identify the people we’re trying to reach.
In 2022, the final pre-election Marquette poll came pretty close to capturing the actual election results in the two big races in Wisconsin for governor and Senate. Was that an argument for continuing to operate as you had done in the past?
Franklin: Yeah, in 2022, our average error was 2.2 percentage points on the margin in the Senate and governor races and that was right at our long-term average since 2012. So it wasn’t that 2022 was a bad year. It was a good, average year, but what we did find during 2022 is it was harder to reach people during the normal interviewing period, and some groups in particular, such as voters under 30, became noticeably harder to reach in 2022.
What kind of response rates have you been getting?
Franklin: Our response rates in telephone polls last fall are closely in line with national averages across polls but are stunningly low: 1.5 percent of all calls dialed that result in an interview. When we started the poll in 2012, that was around 6 or 7 percent.
So, is the issue more one of cost, one of how long it takes to get enough interviews, or accuracy concerns — or all of the above?
Franklin: I think our accuracy has remained pretty good, so that’s not the fundamental driver here. But costs have certainly gone up because it has taken a lot more hours of interviewer time to reach the same number of respondents … and while the sample has not been far off, in certain groups, like voters under 30, reaching them on the telephone has become increasingly difficult. When you think about the number of people who don’t answer their phone under any circumstances, it becomes increasingly desirable to reach them by other means.
So, what are the most important changes in the poll?
Franklin: The most important thing probably is that we’ve always had a sample of registered voters, but we called people at random and asked them if they are registered (to vote) and used that as our sample.
What we are switching to is a list of registered voters based on the (official) Wisconsin registered voters list … We’re moving to this listed sample which political pollsters have been using for a good while, but because I didn’t think the old method was broken, I resisted changing to it. But now the advantage is we’ll know that people from that registered voter list really are registered, and we can contact them knowing something about them because of information that’s on the voter list.
We are also contacting people from a separate panel of respondents who are asked to participate in surveys through letters sent to their home addresses. And they make up about a quarter of the sample while the registered voter list makes up three-quarters of the sample. The (panel) group has agreed in the past to do surveys so we can get a very high response rate from them.
In the old poll, all the interviews were conducted over the telephone, and that’s changing.
Franklin: That’s very much the change … now we are inviting people to conduct the survey online, but if they don’t want to or can’t be reached online, then we call them. So, it’s a blend of online and telephone. In this (new) sample, 666 interviews were completed online and 247 were done by the phone … The polling industry has moved very much toward online surveys.
What is the state of anxiety among pollsters right now?
Franklin: The state of anxiety is always high in polling, especially before elections. I think the major concerns are, are we getting a representative sample of the public or of voters? And that means concerns about under-representing certain groups, whether that’s younger voters or whether that’s Trump voters … The shift to the new methods we’re using gives us a little bit more information about everyone … so that we can better assess any bias coming from people not responding to the survey. In our old method, we knew almost nothing about the people that did not respond to us.
What is your best advice to people who follow your survey about how to be a smart consumer of polls?
Franklin: First of all, the most important thing is, don’t panic over any particular result. All surveys have a margin of error.
When you look at the lifespan of the Marquette Poll, what are the biggest changes in voters you’ve measured since you started polling in January of 2012?
Franklin: I think the most important stand-out is the shift of the state in a more Republican direction based on party identification. In 2012, we were at about 31 percent Democratic, 27 percent Republican (and the rest independent) and now we’re about 29 percent Republican and 28 percent Democratic. It’s about a one-point (Republican) margin. Almost all of that comes from a rise in Republican identification among white men without a college degree who’ve moved from 4 points more Republican than Democratic to about 25 points more Republican than Democratic over this time. We were already a competitive state but now we’re an even more evenly balanced state … (Also) If you look at our polling over time, which is on the website, you notice that most questions are pretty stable over the months and years (but) some move in a pretty meaningful direction. Since 2012, support for concealed carry (of firearms) has gone up, support for same-sex marriage has gone up and support for marijuana legalization has gone up. Those have all moved in gradual but consistent directions.
What are the biggest takeaways from your polling in Wisconsin on abortion both before the 2022 Dobbs decision overturning Roe v. Wade and since the Dobbs decision?
Franklin: Opinion in the state on abortion has been pretty remarkably stable over the last 11 years. We have consistently seen around 60 to 65 percent who say abortion should be legal in all or most circumstances and about 30 to 35 percent who say it should be illegal in all or most (circumstances) … Since the Dobbs decision, it looks like there is a little bit of decline in the percentage who say it should be illegal in all circumstances.
So, more polls to come in 2023 and a lot more in 2024?
Franklin: That’s exactly right. With this shift in methodology and sample design and method of doing the interviews, we’re using this (new) June survey and the one coming up in the fall to test these changes and make sure we understand whether they’ve mattered and how they may have mattered, so that by the time we get to 2024 we will be confident that we understand those changes and they’re doing what we want them to, giving us a better sample and giving us more confidence in our results going into the election year.
Craig Gilbert provides Wisconsin political analysis as a fellow with Marquette University Law School's Lubar Center for Public Policy Research and Civic Education. Prior to the fellowship, Gilbert reported on politics for 35 years at the Journal Sentinel, the last 25 in its Washington Bureau. His column continues that independent reporting tradition and goes through the established Journal Sentinel editing process.
Follow him on Twitter: @Wisvoter.
This article originally appeared on Milwaukee Journal Sentinel: Marquette Law poll debuts new methodology in survey released today
