NZ public poll methodologies

June 10th, 2014 at 10:00 am by David Farrar


Andrew at Grumpollie has put together this very useful table showing the different methodologies of the five public pollsters (I don’t count Horizon) in NZ.

Russell Brown at Public Address noted:

One big advantage for the political Left of John Banks’ sorry experience with the courts last week is that it meant people weren’t talking about the Left’s really awful result in the latest Roy Morgan poll.

Morgan has National up seven points to 52.5% support, and Labour and the Greens both down to a combined 38%. The Greens shed 4.5 points to slump to 9% support, their lowest level since 2011. This will hurt at The Standard and the Daily Blog, where Roy Morgan polls and their inclusion of mobile phones are something of an article of faith.

It’s possible that this is an outlier poll — it does, after all, show Act doubling its support — but while Gary Morgan’s commentary on the results is typically bonkers, there’s nothing in particular wrong with the company’s methodology. And, significantly, the swing is reflected in the regular Government Confidence Rating (whether New Zealand is “heading in the right direction” or not.) It simply looks like a very healthy post-Budget poll for National.

But a friend put another interpretation to me on Friday: that the public has had a look at Internet-Mana and decided a potential centre-left coalition is really not to its taste. Perhaps Labour has internal polling to similar effect, explaining the spluttering reaction of of a number of Labour MPs to the prospect of cooperating with the party of Kim Dotcom and Laila Harre.

The commentary on the Roy Morgan polls is generally hilarious, and somewhat removed from reality. This doesn’t mean their polls are inaccurate.

However what Andrew’s table shows is that we know very little about how they conduct their polls – which would help people make a judgement on reliability.

The other four pollsters have signed up to the NZ Political Polling Code. This requires signatories to publicly release significant aspects of their methodologies. This is an important step for transparency. Roy Morgan has not signed up to the code, and we don’t know a lot about how their polls are done. We don’t even know if they weight the polls to the NZ adult population.

This doesn’t mean their polls are wrong, just as it doesn’t mean pollsters who have signed up will always get it right. For example a poll I did on attitudes to smoking and lung cancer found a lower prevalence rate for smoking than the census. Now the census figure is almost certainly the more accurate, so the difference may be down to how people respond to a phone poll vs a census, or it may be that even with weighting we under-surveyed current smokers. Good pollsters will always be critiquing their own methodology and considering how to enhance or review it.

It would be a very good thing if Roy Morgan did release more information on their methodology, so people can understand their results better in the right context.

The worm

November 21st, 2011 at 10:00 am by David Farrar

The Herald reports:

Expect even more relentless repetition of upbeat language than usual from Prime Minister John Key and Labour leader Phil Goff as the worm returns for tonight’s leaders’ debate on TV3.

The debate will use the Roy Morgan Reactor, more commonly known as “the worm”, to measure the response of a studio audience of undecided voters selected for a balance of gender and age.

TV3 and Roy Morgan invite viewers who have an iPhone, Android mobile, iPad or tablet device to participate.

You can download the reactor or worm here.

Personally I think such things are gimmicks that distract people from the substance of the debate. My hope (a forlorn one probably) is that the media don’t make the worm the headline the next day, but actually you know listen to the debate themselves and form their own opinions as to the strengths and weaknesses of the the leaders arguments. The result of the worm should be a mention in the story, but not the core focus of it.

The fact that anyone can download the worm, means that the results may have nothing to do with the debate, and everything to do with people’s pre-extsing opinions. I really can’t see a lot of undecided voters caring enough to download the worm – the partisans will.

And even the studio audience of undecided voters can be less than balanced. They may be undecided between Labour and Greens, or between Maori and Mana. In my opinion they would be better to have the studio audience comprised of “swinging voters”, being voters who have voted (for example) for both a CR and a CL party in the last few elections. An undecided voter tends to be anti-Government because the Government is the known quantity, and they are deciding which of the alternatives to vote for.

So my point is, yes the worm may be a lot of fun. But please do not let the results of the worm be represented as a scientific poll, and do not let it replace your own judgement about the debate. Tune into TV3 and watch for yourself, rather than rely on second hand reports.

Roy Morgan and NZ First

November 12th, 2011 at 11:39 am by David Farrar

There is no doubt NZ First has increased its support. It usually does when they get in the news, and ironically as much as Winston loves to pretend there is a media conspiracy against him, they in fact give him and NZ First far more publicity (but not scrutiny) than other parties polling at his level.

So they are up in the polls, but are they on the verge of 5% as Roy Morgan had them? No, or not yet, in my opinion.

Roy Morgan often has had NZ First higher than the other polls. They had them at 4.5% in May 2011, 5% in April 2011, 5.5% in Jan 2011, 4.5% in Sep 2010, 4.5% in Aug 2010, and prior to the last election had them twice at 6.5% in July and Aug 2008.

As always, I recommend looking at the average of the polls. This had them at 2.2% in October and rising to 3.1% in November. So as I said an increase, but still around 50,000 votes short of 5%.

Invest in the polls

August 31st, 2009 at 4:58 pm by David Farrar

iPredict has six new stocks – polling stocks. And every fortnight they will pay out and new ones occur. Basically you can now invest in what you think the next Roy Morgan poll will say for National and Labour. The stocks are:

  1. National to fall
  2. National to not change
  3. National to rise
  4. Labour to fall
  5. Labour to not change
  6. Labour to rise

The current stocks are for what the Roy Morgan polls for the period 17 to 30 August will do in comparison to the results for the previous period of 2 to 16 August.  Normally I would expect the poll to 30 August results to come out late Friday, so you have four days to invest – and hopefully make a return.

So if you have a view about how National or how Labour has done in the last fortnight – you can now put your money where your mouth is.

Post-honeymoon bliss?

June 21st, 2009 at 9:20 am by David Farrar

During late May and early June we had trumpeted that the honeymoon is over for the Government with the “week from hell” over Christine Rankin, Mt Albert mistakes, the Richard Worth affair, cancelled tax cuts etc.

Yet the Roy Morgan poll taken from 1 to 14 June finds no significant change in support for the Government and Opposition. National leads Labour by 19%. Also an increase in those saying NZ is heading in the right direction.

So if the honeymoon is over, I’d say National will be pretty happy with those post-honeymoon numbers.

ISP Performance

April 17th, 2008 at 2:22 pm by David Farrar

Roy Morgan has survey results on ISP performance.

They don’t cover all ISPs, but of the ones they do, Actrix tops the satisfaction survey at 86% followed by Paradise at 76%. Down the bottom Ihug is on 64%, Woosh 60%, and Xtra 55%.