When polls go wrong

Most of the time, most polls accurately predict elections outcomes within the margin of error. The polls of polls predicted all 50 states in the recent US presidential election.

But sometimes you get an election result which was contrary to not just some or even most of the polls – but all of them.

These are always fascinating to pollsters, as they provide great learning experiences. Some of the more well known poll disasters was not picking the defeat of Jeff Kennett in Victoria and one of the Obama v Clinton primary battles.

We can now add to that the recent British Columbia elections. Huff Post reports:

It was a historic, completely unexpected comeback.

After trailing in the polls for more than a year, often with a deficit of more than 15 points, the B.C. Liberals under Christy Clark managed to win re-election last night. And they did so easily, with 44.4 per cent of the vote against 39.5 per cent for the NDP (a wider margin than the one that elected Gordon Campbell in 2009) and 50 seats, more than the 45 seats her party occupied when the legislature was dissolved and the campaign got under way.

Put simply, the polls got it spectacularly wrong.

How wrong? Another story states:

A May 10 Angus Reid poll showed that 45 per cent of 803 voters surveyed intended to support the NDP, while 36 per cent said they would vote for the Liberals. That was a nine-point overall lead over the Liberals. An earlier Ipsos Reid poll, which surveyed 800 adult British Columbians, found that 43 per cent of surveyed voters were supporting the NDP.

An Ekos poll, with robocall technology, on Monday gave the NDP 40.5 per cent of voter support. …

But almost two hours after polls have closed, the Liberals have 44.7 per cent of the vote, the NDP with 39.1 per cent and the Liberals leading or elected in 52 ridings, with 43 needed for a majority.

19 polls in April and May showed the NDP ahead and the most recent gave them a 9% lead. They lost by 5%. What went wrong? Was it that most of them were online polls?

A further story:

That every polling firm in the field, using a mix of methodologies, was unable to get a good result (and they mostly showed consistency even at the regional levels) suggests that something systemically wrong was taking place in their sampling methods. Are pollsters not building a sample that is reflective of the broader population anymore? Are they not polling those who actually vote? Are people no longer responding to polls truthfully? Do the now ubiquitous online panels and automated telephone polls have intrinsic limitations that can be amplified under certain circumstances (both have had success, and failure, in the past)?

These questions will need to be answered. An almost literal last-minute swing in voting intentions worth about 13 points does not seem to be plausible. The effect of low turnout, and the inherent discrepancies it can cause in polling, may be a place to start.

So why were the polls all wrong? It seems it was turnout? The Globe and Mail report:

Polls gave the NDP a two-to-one lead over the Liberals among voters 18 to 34 years of age, Mr. Canseco said.

“If that young vote decides not to show up, you’re kissing goodbye to a third of your base, and that’s exactly what happened,” he said, noting that the overall turnout was “abysmal” (52 per cent). “When you have a party at 45 per cent, and they end up with 39, that means there was a difficulty getting their voters out.”

Polls are based on stated preferences of the general population, not those who actually show up to vote, Mr. Canseco said. “The electorate did not resemble the electorate we were polling.”

I don’t know the methodology of the Canadian pollsters but one should always try to determine how likely a respondent is to actually vote, and eliminate those unlikely to vote.

Ipsos Canada comments:

In British Columbia, we interviewed 1,400 voters on Election Day and, as you’ll see, the numbers virtually matched the real outcome in terms of voter preference.

So the exit poll was accurate.

But it also tells a story as to why this happened right down to the last minute. The reality is that one in 10 (11%) BC voters decided in the voting booth on election day to mark their ballot for their candidate—and with one of the lowest turnouts in provincial voting ever (52%) it was motivated voters, Liberals, who bested the NDP in the voting booth.

The long and the short of it was that NDP voters did not get out and fulfill their promise to vote for the party of their choice – they stayed home while Liberal voters showed up. As such, a small number of voters were able to influence the greater outcome.

The Greens have this issue also in NZ. Their voters tend to be younger and tend not to turnout, hence why their results tend to be below their poll ratings.

In fact, nearly one-quarter (23%) of voters said they decided who they were going to vote for in the last week of the campaign. So the trend had continued from the week previously and these late deciders chose to vote BC Liberal by a 7 point margin over the NDP (41% BC Lib vs. 34% NDP).

Possibly the late deciders were fearful of change.

It’s clear that the negative-advertising campaign of the Liberals waged against the NDP had a slaughtering effect. If ever there was a case to behold that negative advertising campaigns work, it is here where the Liberals were able to take the NDP lead at the outset of the campaign of 20+ points in some of the polls and put it in the hole. The following show the changes in what happened in the final days of the campaign:

The NDP are quite like the Greens (hard left). Imagine what a good election campaign can do with the thought of a Green/Labour Government!