About
Contact
Advertise

Facebook
Twitter

June 12, 2019 7:00am by David Farrar

Why the polls diverged

A good article in Stuff (I was interviewed for it) which covers the divergent poll results, and associated issues.

Share this:

Facebook
Twitter
LinkedIn
Reddit
WhatsApp
More

Pinterest
Print
Tumblr

Like this:

Like Loading...

Email
Facebook
Twitter
Comment45

NZ Politics
Polls

Related Stories

538’s 13 tips for reading polls
Clinton’s lead bigger with live polls
How the pollsters did provisionally

Comments (45)

Login to comment or vote

Lee C

It’s an interesting read. I’m often likely to go into ‘knee-jerk’ mode when I see/hear polls, believing them to have specific agendas for publishing their results, or being run based on the specific agenda of their client.
What is interesting is the impact of confirmation bias and how the media can collectively engineer consent. We see it with the likes of Tova O’Brian all the time.
In a schoolyard, it would be called peer-pressure, even bullying. And it appears that pollsters can fall under its sway – hence the skewed reporting and reinforcement of a specific ‘narrative’ in both the USA and Australia.
40 0
Log in to Reply
Report
June 12, 2019 7:10am
- David George
  
  We have a Prime Minister that feels compassion, unfortunately she is naive/foolish enough to believe her “feelings” are a moral good, always the right approach to everything.
  I’m sure she believes that vile fascist dictators like Xi Jinping, or dangerous criminals or lying fake illegal immigrants just need to be treated with “kindness”.
  It’s like having a sixteen year old running the show.
  
  33 1
  Log in to Reply
  Report
  June 12, 2019 8:06am
  - cmm
    
    As she has said, she has a wonderful intuition. She does not pollute her grey matter with data.
    
    That explains why Kiwibuild is in a mess. You need to use spreadsheets and thinking to manage construction – not feelz.
    
    The same with planting a billion trees. The soundbite is the easy bit. Actually getting the logistics together is the hard bit.
    
    30 0
    Log in to Reply
    Report
    June 12, 2019 8:22am
    - Farmer boy
      
      Yes. We have a rookie PM who puts feelings above business logic. This is her strategy; it is also her glaring weakness. Detail is all to hard for Jacinderella. Implementation is completely beyond her – as evidenced by her pointless visit to Paris. Jacinda, the headline girl.
      
      19 1
      Log in to Reply
      Report
      June 12, 2019 8:35am
cmm

“They were taken a week apart, that doesn’t happen in a week.”

Sorry DPF, but it does happen in a week.

Look how fast things swung when Little stepped aside and changed leader.

The age of considered thought is dead because thinking is hard, emotion is easy and thus easy to manipulate.

Everything is now PR managed and the best PR weapon is emotion. Emotional responses are fast and can change very quickly too.

What were the sample sizes?
41 0
Log in to Reply
Report
June 12, 2019 7:24am
- kowtow
  
  Agree.
  Greta Thunberg , pupil marches, massive media coverage ……and now councils ,which should know better and are nominally run by rational adults vote that there’s a climate emergency.
  
  Last night TV1 news ran a slot on tourism and carbon guilt , tonight it’s target is milk. If we were a Soviet republic the editor would be shot for treason/ economic sabotage. “News” it definitely is not.
  
  40 0
  Log in to Reply
  Report
  June 12, 2019 7:37am
  - cmm
    
    Yup, for every journalist there are multiple PR people. These days “journalism” has largely become cut-and-pasting out of press releases.
    
    News should be about informing people so they can make decisions for themselves, but now people are being served pre-digested “news smooties”; a bit of news as well as a whole lot of emotion so you don’t have to think about it.
    
    31 0
    Log in to Reply
    Report
    June 12, 2019 7:55am
- Brian McDonald
  
  What event happened in the week that you think swung the electorate sentiment in a similar fashion to Little relinquishing the LP leadership?
  
  1 11
  Log in to Reply
  Report
  June 12, 2019 8:03am
  - cmm
    
    An emotion bomb.
    
    6 0
    Log in to Reply
    Report
    June 12, 2019 8:34am
    - Brian McDonald
      
      About what?
      
      0 4
      Log in to Reply
      Report
      June 12, 2019 9:23am
      - Tall Man
        
        I realise that the more astute already were aware that she has a problem with the truth but it has been clearly exposed here.
        
        She also campaigned on transparency and pushed a “well being” budget that has been best described as “ho hum” but no one at all thinks it is “well being” except for maybe a few morons and richard branson.
        
        It’s hard for some voters to admit that they voted for “lipstick” but still got the pig.
        
        0 0
        Report
        June 12, 2019 4:00pm
- Gravelroad
  
  On his newstalkzb programme yesterday, Andrew Dickens ridiculed the Auckland City Council’s decision to declare a state of “climate emergency”.
  Seems one of the motivating factors for these electeds was that the PR advisors had said that if they didn’t, it could have an impact on their reputations.
  In other words:
  1. It is local body election year.
  2. Candidates will be asked (frequently) by a section of the electorate and the media if they believe in the need for the declaration of this “emergency “.
  3. As a candidate, you have to say “yes” if you want to be elected/re-elected – otherwise your reputation will be trashed.
  4. It’s OK though because you don’t actually have to do anything. Just say you believe .
  
  19 0
  Log in to Reply
  Report
  June 12, 2019 8:45am
  - Surely Knott
    
    Interesting that it was lead by Hulse (she is chair of the environment committee) and not running for re-election, but where was Phil Goff I couldn’t see him on the video? Surely he should have been there for such an important declaration? And when they closed the Waitakere Ranges forest our largest regional park on claims of widespread Kauri Dieback, using a map with spurious and conflicted data, again it was lead by Hulse who would suffer no political backlash as she is standing down. But where was Phil Goff? He was away on council business.
    
    3 0
    Log in to Reply
    Report
    June 12, 2019 10:07am
- redeye
  
  I thought this was the telling bit;-
  
  Up until the 2017 election, Colmar Brunton exclusively used landline phones, but they have since changed to a 50/50 split of landlines and mobile. Reid Research uses a split of 75 per cent phone and 25 per cent online polling.
  
  8 0
  Log in to Reply
  Report
  June 12, 2019 9:10am
kevn

swarbrick dancing on the head of a pin with Hosking on a legalize cannabis debate.
21 3
Log in to Reply
Report
June 12, 2019 7:47am
- Brian McDonald
  
  Maybe Hosking should interview the NZ Drug Foundation on their view, or the gang leaders who agree with him.
  
  10 3
  Log in to Reply
  Report
  June 12, 2019 8:05am
  - kevn
    
    @brian mc…. I gather the gangs have long moved on to selling the super addictive p.
    One can grow their own weed and I guess many do.
    Yes, Hosking should interview whoever it is from the Drug Foundation.
    
    5 1
    Log in to Reply
    Report
    June 12, 2019 9:15am
  - Tall Man
    
    Anyone who actually supports legalising cannabis is an idiot without any regard for the teenagers that are our future.
    
    Virtually every study clearly indicates that usage will increase after legalisation.
    
    Virtually every medical study finds serious health concerns for younger users.
    
    Every study shows that youth alcohol abuse increased massively after the drinking age in NZ was lowered to 18.
    
    Virtually every study shows the age of teen drinking fell after the drinking age was lowered to 18 in NZ.
    
    Idiots are trying to tell us that our youth will not use more cannabis, will not drive under the influence of cannabis more is either retarded or a liar.
    
    The one stat that really concerned me was the 300% increase in deaths in Colorado road accidents where cannabis was present. Silly swarbrick tried to deflect that stat but she couldn’t present a clear reason why she thought it was irrelevant.
    
    0 0
    Log in to Reply
    Report
    June 12, 2019 3:09pm
  - kevn
    
    @Brian… I hope you listened to Hosking this morning (13th) If not, find it.
    Hosking was speaking to senior Colorado Law Enforcement.
    Downsides all round. Major.
    
    0 0
    Log in to Reply
    Report
    June 13, 2019 10:36am
I Hate Wordpress

The samples sizes were 1000. No matter how good you think you are averaging that out against ~5million people they could still both be “right”.

If we really want to see whether extrapolation or the actual data is the culperate each company should do a poll based on the others raw data.
4 2
Log in to Reply
Report
June 12, 2019 7:59am
- cmm
  
  If you have a truly unbiased sample, then 1000 should give you a 3% margin of error.
  
  The rough numbers for this calculation is:
  1) Take the square root of the number: Sqrt(1000) is about 31.
  2) Divide 100% by that square root. 100%/31 = 3%.
  
  It is not impossible to be outside that margin of error, just unlikely.
  
  1000 is the “sweet spot” number. To get to a 1% margin of error would need 10,000 samples – way more.
  
  The most important factors are still:
  1) How you pose the actual questions. Even asking people if they are having a good evening can influence answers. Talking to people after they have eaten or watched their favourite TV programme changes things too.
  2) How you select the samples: Do you just take the person who answers the phone or try to fit demographics?
  
  12 0
  Log in to Reply
  Report
  June 12, 2019 8:32am
  - Ross12
    
    cmm
    While I agree with your numbers I think it is all academic if the pollsters do not include the “don’t knows” or “undecided” as part of the results. I know I’m sounding like a cracked record on this, because I keep ranting about it, but I think the “undecided” number is as important as Labour or National’s number.
    CB reported separately for this recent poll that number was 14% . Put that in and where do the Greens show up , for example ? What does NZF look like ?
    
    5 0
    Log in to Reply
    Report
    June 12, 2019 9:26am
    - Tall Man
      
      “I know I’m sounding like a cracked record on this”
      
      No you’re not, you are absolutely correct.
      
      I was in Oz during weeks before and after the recent election. It was clear to me that labour were going to loose. Every person that I spoke to told me they hated labour and would vote to make sure they did not get in.
      
      Now, I have to admit that I only spoke to people in a job and receiving little benefit from the state and we have a shitload of non workers here but I think they represent the 25% support labour had and “stardust” ganered them another 12%.
      
      Now, the media has worked overtime to both continue and polish that stardust but the gloss has been overpolished and the turd beneath is starting to show through.
      
      2 0
      Log in to Reply
      Report
      June 12, 2019 3:02pm
  - Crisplion
    
    A slight correction cmm: The margin of error is (about) two standard errors, being the margin within which results will fall 95% of the time.
    
    The standard error is the square root of the proportion of those polled supporting a given party, times one minus that proportion, divided by the number polled. So if 50% of 1,000 people polled say they’ll vote Labour, then the standard error is the square root of 0.5 times 0.5 (0.5 being 50% expressed as a proportion), divided by 1,000, which is about 1.6. Multiplying that by two, we get about 3.2%. So, a similar result to yours, but your formula wasn’t quite right. I’m not pointing this out to be pedantic – bear with me:
    
    Very close to 95% of all results will fall within two standard errors of the true proportion (the standard ‘margin of error’ given by pollsters. But 99.7% of results fall within three standard errors and the percentage falling within five standard errors is 99.99997. So only about one in three and a half million results will fall outside that band.
    
    Now the two results for Labour were 8.8 percentage points apart, which is 5.6 standard errors with a standard error of 1.6 points. That means that while it’s possible that error of measurement was the only reason that the polls diverged by the margin that they did, it’s very, very unlikely.
    
    So I can’t agree with DPF’s analysis in the Stuff article. I think sample bias pretty much has to be at work, and the different sampling methodologies is the prime suspect. Which is right? Probably neither but we can’t know for sure. Damned lies, statistics and all that…
    
    5 0
    Log in to Reply
    Report
    June 12, 2019 9:57am
    - cmm
      
      Correct.
      
      The square root thing is just a quick rule of thumb “sniff test”, very useful for bullshit detection.
      
      It’s also something that is pretty easy to remember. It’s a long time since I opened my statistics text books at university, about 38 years now!
      
      1 0
      Log in to Reply
      Report
      June 12, 2019 12:32pm
    - Rachael Membery
      
      How often do two polls come out at the same time, other than at election time that is?
      
      0 0
      Log in to Reply
      Report
      June 12, 2019 1:47pm
- Nigel Kearney
  
  Easy to say ‘one must be wrong’. Not so easy to define what ‘wrong’ means. 95% seems like a lot, but polls are taken fairly frequently you’d expect that if 1 in 20 polls were this far off, we would be having this discussion more often.
  
  Even more so when it is the difference of two polls. If one poll is more than 1 standard deviation above the true mean and the other is more than 1 below, the difference is outside the confidence interval of either poll, but the polls themselves are well within the 95% CI.
  
  Add in the time difference in polling and it all adds up to nothing much, as far as I can tell.
  
  4 0
  Log in to Reply
  Report
  June 12, 2019 11:07am
hullkiwi

Polls being wrong is nothing new. The UK has a history of wrong polls. In 1992, polls predicted a Labour win under Kinnock, polls had been predicting a Labour victory for months. But like Australia this year, the unexpected happens, the Tories won. After that they went into navel gazing to work out what they had done wrong.

The biggest factor had been “herding”, no body was prepared to predict anything, but a Labour win.

Our problem here is a poor corporate commercial/advertisement, market driven media who do not report news or mature commentary and this skewing what the sheeple read/watch/listen, and what they think.
18 0
Log in to Reply
Report
June 12, 2019 7:59am
- Scott
  
  Hull Kiwi – agree with you on the importance of herding and groupthink. It leads to the phenomenon of the “shy Tory” who doesn’t want to reveal to the pollster that they are voting for Brexit or Donald Trump or Scott Morrison or those that are not favoured by the chattering class. So they don’t actually tell the pollster who they are going to vote for. They just quietly get into the voting booth by themselves and vote for the non-politically correct candidate. They just aren’t prepared to tell anyone about it.
  
  The other issue in the polls and reporting on the polls is wishful thinking. Prior to the 2016 US presidential election all the pollsters were confidently picking a victory for Hillary Clinton. It is also true that they wanted Hillary Clinton to win. As did 99% of the mainstream media. So they saw what they wanted to see.
  
  10 0
  Log in to Reply
  Report
  June 12, 2019 8:40am
emmess

Simple, Newshub biased the poll but asking about ‘the Hack’ beforehand.
The MSM didn’t mention that.
26 0
Log in to Reply
Report
June 12, 2019 8:02am
- G152
  
  Find the questionnaire.
  Carefully read the questions and see the bias and which way it will produce a result that the pollster wants…
  Remember those polls that projected something like a 90% win for a certain American wannabe president
  
  8 0
  Log in to Reply
  Report
  June 12, 2019 8:34am
RRM

Don’t forget, everyone on the right is Hitler.

Social media has long been afflicted with that, and the news has gone the same way.
25 0
Log in to Reply
Report
June 12, 2019 8:19am
- cmm
  
  And remember Hitler was a socialist.
  
  15 0
  Log in to Reply
  Report
  June 12, 2019 8:33am
- MCos
  
  I can understand people lying to pollsters out of bloodymindness, but not because they’re somehow embarrassed to admit they favour one side or the other. After all the two people on the line don’t know each other.
  
  1 3
  Log in to Reply
  Report
  June 12, 2019 10:11am
  - Rachael Membery
    
    And there is nothing to say that they record accurately either, there is an awful lot of faith that the person taking the poll isn’t projecting bias.
    
    I have been polled once and I could tell that the person polling me didn’t like my answers.
    
    2 0
    Log in to Reply
    Report
    June 12, 2019 1:52pm
scrubone

Is it just my imagination, or did that report spend a lot of time explaining that National can’t be in the lead, but ignored the other possibility?
5 0
Log in to Reply
Report
June 12, 2019 8:25am
lastmanstanding

No mention of the “shy Tory” effect yet we saw they play out in the US in 2016 and in Australia in the recent election.

In both cases the Left ahead until Polling Day.

What about the dont knows. I understand in both Polls they were north of 10%. Thats a lot.

I suspect the gap between the Polls and the reality is going to grow.
10 0
Log in to Reply
Report
June 12, 2019 9:12am
- davidp
  
  Shy Tory is a thing. In Australia recently we saw a worker sacked for asking Shorten a question that made him uncomfortable. And in NZ we’ve seen Ardern, Robertson, and Peters set the police on someone embarrasing the government. I’d be reluctant to tell a pollster I support National, and risk finding out that that they’re polling on behalf of Labour or a left wing group and find I’d be doxed, sacked, or being investigated for a hate crime where “hate” means disagreeing with Ardern.
  
  4 0
  Log in to Reply
  Report
  June 12, 2019 11:22am
KevO'Brien

The polls are mathematical nonsense. Random sampling is not there. Like climate change the data is inferred.

A real measure of turning against the COL is the journalists freshly criticising them. That is increasing. People are getting wary too of freely giving their opinions. Changes are afoot. Expect that when a bunch of amateurs got selected for the most responsible job in NZ.
☑️ ☑️
11 1
Log in to Reply
Report
June 12, 2019 9:14am
- tf78
  
  We need to be brave and say exactly what we think, it only takes a few to lead the way and set the world straight again. Remain silent and you will be silenced forever.
  
  4 0
  Log in to Reply
  Report
  June 12, 2019 11:09am
  - Tall Man
    
    Try it and expect a visit from an armed police officer asking you why you think the President of the US is a good guy..
    
    0 0
    Log in to Reply
    Report
    June 12, 2019 2:51pm
    - tf78
      
      Because he is.
      
      1 0
      Log in to Reply
      Report
      June 12, 2019 5:44pm
      - Tall Man
        
        But you can’t think it 😉
        
        0 0
        Report
        June 12, 2019 7:01pm
Jellybean

I have been wondering about the Judith Collins result in both polls. Although I know some National supporters who would like a different leader, most of them struggle to say who it should be and almost none of them think it should be JC.

My theory? Labour and Green voters are trying to create a story and destabilise National’s leadership by answering JC for preferred PM. That would explain some of the difference between Ardern’s preferred PM and the total party result for Labour and Greens. All the Labour and Green supporters I know just LOVE Ardern.

Would be interesting to see an analysis showing what preferred party answers were given by those who gave JC as preferred PM.
5 0
Log in to Reply
Report
June 12, 2019 1:39pm
- Tall Man
  
  My thought as well.
  
  Plus, don’t knows are huge.
  
  2 0
  Log in to Reply
  Report
  June 12, 2019 2:50pm

Add a Comment Cancel reply

Login to comment or vote

« If you can afford a $650,000 deposit you don’t need a subsidy

General Debate 12 June 2019 »

Donations

You can now donate to Kiwiblog

Exclusive content from DPF on Patreon

Become a Patron!

Highlights

The Kiwis are fleeing
Inflation by term
Houston we have a problem
Our immunisation disaster

Top Stories

General Debate 18 April 2024
General Debate 19 April 2024
This could explain the brickbat
No excuse for fleeing
Two terrible attacks in Australia
Singapore takes defence very seriously
We still have an inflation problem
Are those with access cards the problem?
UK Labour Deputy Leader in trouble
Ministry of Education Cuts in Perspective.

Recent Posts

Are those with access cards the problem? 19 April 2024
Singapore takes defence very seriously 19 April 2024
General Debate 19 April 2024 19 April 2024
Two terrible attacks in Australia 19 April 2024
No excuse for fleeing 18 April 2024
This could explain the brickbat 18 April 2024
General Debate 18 April 2024 18 April 2024
We still have an inflation problem 18 April 2024
UK Labour Deputy Leader in trouble 17 April 2024
Ministry of Education Cuts in Perspective. 17 April 2024
Graham Adams on MACA and the Court of Appeal 17 April 2024
General Debate 17 April 2024 17 April 2024

Never miss a post

Email Address

Recent Comments

Are those with access cards the problem?
the deity formerly known as nigel6888
Are those with access cards the problem?
WorthwhileAucklandInitiative
General Debate 19 April 2024
Maggie Pie
Are those with access cards the problem?
70andstilllearning
General Debate 19 April 2024
fightingtemeraire
Singapore takes defence very seriously
NeverMindTheBollocks
General Debate 19 April 2024
julesnzuk
Singapore takes defence very seriously
NeverMindTheBollocks
General Debate 19 April 2024
greybeard
Singapore takes defence very seriously
NeverMindTheBollocks
Are those with access cards the problem?
Maggy Wassilieff
General Debate 19 April 2024
imalwaysright
Are those with access cards the problem?
NeverMindTheBollocks
Singapore takes defence very seriously
Geri Sjovik
General Debate 19 April 2024
Sparrow

Links

A who's who of New Zealand's dodgiest companies
No Right Turn
Point of Order: Buzz from the Beehive - 19/4/24
Breaking Views
"Under a better Resource Management system [sic], there would be no need for fast-track approval processes."
Not PC
Daily review 19/04/2024
The Standard
On Lee’s watch, Economic Development seems to be stuck on scoring points from promoting sporting events
Point of Order
Word of the day
Hompaddock
Scottish Skullduggery
No Minister
1 Key Rule to Prevent 50% of Your Chess Mistakes
Utopia
Again, hate crimes are not necessarily terrorism.
Kiwipolitico
Te Papa gutlessly cower to Chief Culture Censor Paul Goldsmith
The Daily Blog
Despair - construction consenting edition
Offsetting Behaviour
RACIST ADVERTISING
No Punches Pulled (Bob Jones)
Is the state coming for churches?
Bob McCoskrie's McBlog
Super Rugby Predictions for Week 9
Stats Chat
Coalition Promises
NZCPR
Tougher than that
Croaking Cassandra
ABs stripped of 2011 title after try disallowed
Imperator Fish
Legal Beagle: Election '23: The Special Votes
Public Address
Making this movie might have been a mistake
Liam Hehir
Why I am retiring from political blogging
Kiwi FireWalker
Angie Warren-Clark: Meet the Newbies
Dan Moskovitz

editor@kiwiblog.co.nz

PO Box 12270, Thorndon, Wellington 6144

Facebook
Twitter
FourSquare
Instagram
LinkedIn
Google+

© 2003-2024 David Farrar

Archives
Disclosure Statement
Editorial Policy
Privacy Policy
Complaints Policy
Comments Policy

Design by cre8d

%d