FiveThirtyEight generally takes an inclusive attitude towards polls. Our forecast models include polls from pollsters who use traditional methods, i.e., live interviewers. And we include surveys conducted with less tested techniques, such as interactive voice response (or “robopolls”) and online panels. We don’t treat all polls equally — our models account for the methodological quality and past accuracy of each pollster — but we’ll take all the data we can get.
This split, however, between live-interview polls and everything else, is something we keep our eye on. When we launched our general election forecasts in late June, there wasn’t a big difference in the results we were getting from polls using traditional methodologies and polls using newer techniques. Now, it’s pretty clear that Hillary Clinton’s lead over Donald Trump is wider in live-telephone surveys than it is in nonlive surveys.
We don’t know exactly why live-interview polls are getting different results than other types of surveys; there are a lot of potential causes and it’s something we’ll be digging into.
It may be the US version of Shy Tories. People are more willing to say they will back Trump to a machine than to a live person.
As of Tuesday morning, Clinton led Trump by 6 percentage points and had a 79 percent chance of winning, according to our polls-only forecast. But running our polls-only model using only live-interview surveys, Clinton leads Trump by 7 points and has an 86 percent chance of winning. Running it with only nonlive-interview polls, Clinton leads Trump by 5 points and has a 71 percent chance of winning.
So around a 2% difference. They all have Trump well behind but by 5% instead of 7% non-live vs live.
As the cases of Utah and Kansas suggest, I’d put more faith in the live-interview polls than in other types of surveys, all else being equal. Indeed, our forecast models do just that.
I tend to prefer live-interview polls also as the more reliable. But worth noting in the Brexit vote, the online panel polls were more accurate than the phone polls.