Believe it or not, companies are already starting to poll for the 2020 Presidential elections. This time around, nobody knows what to believe when it comes to polls. Have they made real changes since the 2016 debacle? We decided to check in with one of our favorite economists and statisticians John Johnson to keep it real.
Sharyl: Looking back at 2016, it's commonly said the polls were wrong. I'm not sure that's true. What is your hindsight on that?
John Johnson: The national polls did pretty well on the popular votes, within about 1 percent of what actually it was, but the state polls were where there was a real problem. The first issue was, just underestimating, or people refusing to say who they were going to actually vote for. That was a much wider phenomenon in 2016 than prior years. Second, was a lot of late-deciding voters, and the direction of those voters was overwhelmingly for Trump. Then the third was just a flaw in the sampling, where it tended to over-sample Democratic voters, particularly college-educated white voters, were way over-sampled.
Sharyl: That's one thing I found when I was looking inside the polls in 2016. If you looked at who they took polls of, almost everyone polled not just a few more Democrats, but many more Democrats than Republicans.
John Johnson: The problem is, if you happen to over-sample college-educated voters, who happen to be more likely to vote Democratic in a particular election, and then you don't weight that properly, you're going to get the wrong answer. So the consequence of not reporting that is, you might miss a big sub-set of voters that were actually very important, and in this case, definitive.
Sharyl: There was sort of broad agreement that whatever went right or wrong, something had to be changed to be more accurate and credible next time. Have changes been made since 2016 by major polling agencies?
John Johnson: So one thing that has changed, pretty much across the board, is now, in today's environment, about 40% of people vote early, or by absentee. So, how do you traditionally poll? Traditionally we use exit polls, which are literally what they sound like. Somebody stands outside of 500 polling places and tries to count every 10th person, and say "Hey, who did you vote for?" and tries to scribble down on paper who did you vote for and what your characteristics are.
Sharyl: And assuming they tell the truth.
John Johnson: Right, assuming they tell the truth. So there's inherently, at this individual level of exit polling, a lot of noise, and hard to get really large samples of people at any particular area. So there are two news organizations, AP and FOX News, that have partnered with the University of Chicago's opinion research center, NORC, to develop a new survey called AP Vote Cast. They’re going to do a combination of online and telephone surveys, and they're not doing exit polling at all. The timing of the surveys are the four days before election day, up to and through election day; and there's sort of two sets of voters or non-voters they're trying to capture.
Sharyl: And why do you care what “unlikely” voters think?
John Johnson: Well, so the rationale at least, is, that you can start to see who are the disaffected voters. Who are the people that may not show up or didn't show up at the polls. Because sometimes it's not just turn out who came, but it's the people that stayed away that actually determines an election.
Sharyl: It seems to me some news organizations, or even advocates, aren't looking to the polls to measure what public opinion really is. But they're looking to use the polls to sway public opinion, to convince people that things are going a certain way, and doesn't that enter the mix in a way that kind of can skew everything?
John H. Johnson: Well absolutely. We have a lot of misinformation in the data we consume every day. People use numbers to sort of advocate for positions, and of course polling is one of those things. You’ve got Republican and Democratic operatives on both sides trying to use the polls to advance their narratives. You've got reporters who are trying to tell stories, and sometimes intentionally, sometimes not intentionally, just interpreting the numbers through their lens. The biggest thing is, remember, every time you see a number, on TV, in the news media, stop, think, "Does that make any sense? Where is that number coming from? Or is someone telling me a message that they want me to hear?" You have to be vigilant when you rely on polling, and statistics in general.
Johnson says the polling in the 2018 midterms was relatively accurate. The big test is still to come.