Kiwiblog

Most of the time, most polls accurately predict elections outcomes within the margin of error. The polls of polls predicted all 50 states in the recent US presidential election.

But sometimes you get an election result which was contrary to not just some or even most of the polls – but all of them.

These are always fascinating to pollsters, as they provide great learning experiences. Some of the more well known poll disasters was not picking the defeat of Jeff Kennett in Victoria and one of the Obama v Clinton primary battles.

We can now add to that the recent British Columbia elections. Huff Post reports:

It was a historic, completely unexpected comeback.

After trailing in the polls for more than a year, often with a deficit of more than 15 points, the B.C. Liberals under Christy Clark managed to win re-election last night. And they did so easily, with 44.4 per cent of the vote against 39.5 per cent for the NDP (a wider margin than the one that elected Gordon Campbell in 2009) and 50 seats, more than the 45 seats her party occupied when the legislature was dissolved and the campaign got under way.

Put simply, the polls got it spectacularly wrong.

How wrong? Another story states:

A May 10 Angus Reid poll showed that 45 per cent of 803 voters surveyed intended to support the NDP, while 36 per cent said they would vote for the Liberals. That was a nine-point overall lead over the Liberals. An earlier Ipsos Reid poll, which surveyed 800 adult British Columbians, found that 43 per cent of surveyed voters were supporting the NDP.

An Ekos poll, with robocall technology, on Monday gave the NDP 40.5 per cent of voter support. …

But almost two hours after polls have closed, the Liberals have 44.7 per cent of the vote, the NDP with 39.1 per cent and the Liberals leading or elected in 52 ridings, with 43 needed for a majority.

19 polls in April and May showed the NDP ahead and the most recent gave them a 9% lead. They lost by 5%. What went wrong? Was it that most of them were online polls?

A further story:

That every polling firm in the field, using a mix of methodologies, was unable to get a good result (and they mostly showed consistency even at the regional levels) suggests that something systemically wrong was taking place in their sampling methods. Are pollsters not building a sample that is reflective of the broader population anymore? Are they not polling those who actually vote? Are people no longer responding to polls truthfully? Do the now ubiquitous online panels and automated telephone polls have intrinsic limitations that can be amplified under certain circumstances (both have had success, and failure, in the past)?

These questions will need to be answered. An almost literal last-minute swing in voting intentions worth about 13 points does not seem to be plausible. The effect of low turnout, and the inherent discrepancies it can cause in polling, may be a place to start.

So why were the polls all wrong? It seems it was turnout? The Globe and Mail report:

Polls gave the NDP a two-to-one lead over the Liberals among voters 18 to 34 years of age, Mr. Canseco said.

“If that young vote decides not to show up, you’re kissing goodbye to a third of your base, and that’s exactly what happened,” he said, noting that the overall turnout was “abysmal” (52 per cent). “When you have a party at 45 per cent, and they end up with 39, that means there was a difficulty getting their voters out.”

Polls are based on stated preferences of the general population, not those who actually show up to vote, Mr. Canseco said. “The electorate did not resemble the electorate we were polling.”

I don’t know the methodology of the Canadian pollsters but one should always try to determine how likely a respondent is to actually vote, and eliminate those unlikely to vote.

Ipsos Canada comments:

In British Columbia, we interviewed 1,400 voters on Election Day and, as you’ll see, the numbers virtually matched the real outcome in terms of voter preference.

So the exit poll was accurate.

But it also tells a story as to why this happened right down to the last minute. The reality is that one in 10 (11%) BC voters decided in the voting booth on election day to mark their ballot for their candidate—and with one of the lowest turnouts in provincial voting ever (52%) it was motivated voters, Liberals, who bested the NDP in the voting booth.

The long and the short of it was that NDP voters did not get out and fulfill their promise to vote for the party of their choice – they stayed home while Liberal voters showed up. As such, a small number of voters were able to influence the greater outcome.

The Greens have this issue also in NZ. Their voters tend to be younger and tend not to turnout, hence why their results tend to be below their poll ratings.

In fact, nearly one-quarter (23%) of voters said they decided who they were going to vote for in the last week of the campaign. So the trend had continued from the week previously and these late deciders chose to vote BC Liberal by a 7 point margin over the NDP (41% BC Lib vs. 34% NDP).

Possibly the late deciders were fearful of change.

It’s clear that the negative-advertising campaign of the Liberals waged against the NDP had a slaughtering effect. If ever there was a case to behold that negative advertising campaigns work, it is here where the Liberals were able to take the NDP lead at the outset of the campaign of 20+ points in some of the polls and put it in the hole. The following show the changes in what happened in the final days of the campaign:

The NDP are quite like the Greens (hard left). Imagine what a good election campaign can do with the thought of a Green/Labour Government!

April saw four political polls published in New Zealand – a Colmar Brunton, a Reid Research and two Roy Morgan polls.

The average of the public polls has National 12% ahead of Labour – 1% more than in March. The seat projection is centre-right 59 seats, centre-left 59.

Australia has Labor 10% behind the Coalition with four months to go until the election.

In the United States Obama’s ratings have changed little while Hillary Clinton enjoys high favourability ratings.

In the UK the UK Independence Party is now polling well ahead of the Liberal Democrats.

In Canada the Liberals have shot to the lead for the first time in many years, under the new leadership of Justin Trudeau.

The normal two tables are provided comparing the country direction sentiment and head of government approval sentiment for the five countries. New Zealand continues to top both by considerable margins.

We also carry details of polls in New Zealand on the GCSB, Kim Dotcom, asset sales, constitutional issues, public holidays, support for farmers plus the normal business and consumer confidence polls.

This newsletter is normally only available by e-mail.  If you would like to receive future issues, please go to http://listserver.actrix.co.nz/cgi-bin/mailman/listinfo/polling-newsletter to subscribe yourself.

I’ve blogged at Curiablog the results of the latest Roy Morgan poll, which is a big mover. Note, it is often quite variable.

The Herald reports:

In the same poll 48 per cent of voters said internet magnate Kim Dotcom should be allowed to stay in New Zealand – 42 per cent say Dotcom should be sent back to the US, and 10 per cent didn’t know.

This was a poll by 3 News.

I think it was a very bad decision to do a poll on this issue. It is tantamount to doing a poll on whether someone charged of a crime is guilty – before it has even gone to trial!

The decision on extradition is effectively one for the courts. Their role is to determine whether the offences he has been charged with in the US qualify under our extradition treaty and laws.

The decision on extradition is one for the courts, not one for public opinion. Of course Mr Dotcom is trying to make it into a cause celebre, but I am unsure as to why TV3 would play along.

If the courts find Dotcom should not be extradited, then he is very welcome here. If he is extradited and is found not guilty in a trial, then again he is welcome here. But the court processes should be allowed to reach a conclusion.

The Herald reports:

Migrant groups aren’t convinced New Zealand is a racist country, despite a new poll indicating overwhelmingly that it is.

Actually it does no such thing.

Seventy-six per cent of responses gathered by TV3′s The Vote on Wednesday night agreed with the proposition that New Zealand was racist.

That is not a poll. It is a meaningless self-selected sample. The results are in no way representative of the NZ population. It is representative of people who wanted to see a debate on TV3 about if NZ is racist and felt strongly enough to vote on it in their online survey.

There is a place for reporting unscientific surveys such as blog and online polls. But they should never ever be treated as representing NZ public opinion – let alone the basis for an entire news story.

Two polls out tonight. A One News and 3 News poll. They differ significantly.

The weighted average of the last three polls is above. What they show is that the centre-right would have 58 seats, centre-left 60 seats and Maori Party the balance of power with three seats.

Now three polls in as many days have had quite different results – Roy Morgan, Colmar Brunton and Reid Research. The Roy Morgan was over a different time period, so it is less surprising that it has a different result.

The differences between Colmar Brunton and Reid Research are a bit surprising. I regard both companies as good ones, and it is worth pointing out that their results are just within the extremes of the margin of error – ie one may just be at the high end of the normal range and one at the low end. And also recall 1 in 20 polls will fall outside the normal margin of error.

But people often ask when they differ this much, how to decide which one is accurate.

Generally my advice is to average them out. This tends to be pretty reliable, and the recent US elections found the polling averages very reliable.

You can look at how each polling company did in regard to the last election. However I’d be careful about putting too much weight on that, unless a company was way off-beam. Judging the accuracy of a company off one single poll that has a 3.2% margin of error is dangerous. It may be that the final pre-election poll was accurate, but that things changed in the final few days after the poll was taken.

However for those interested the average difference between the polling company’s results and the average result for all parties was

1. Colmar Brunton 0.9%
2. DigiPoll 1.1%
3. Reid Research 1.3%
4. Research International 1.6%
5. Roy Morgan 1.6%

A difference in the average variation by 0.4% is not great, in my opinion. And as I said, I’d be very careful judging off a single poll. In the US they have polls covering scores of elections, so can get a good idea there if a company systemically has one party too high or low.

Now we can’t do that in NZ, but we can look at how each company has rated the major parties over an electoral cycle. Now again be wary of this comparison because over a three year period, the polls will be done at different times. If I get the time, I might try and do a comparison of polls done within say the same week, but for now thought a quick and dirty analysis could be interesting.

Over the 2008 to 2011 period, the average for each polling company and party was:

• Colmar Brunton – National 53.6%, Labour 31.1%, Greens 7.3% = Lead for National of 15.2%
• Reid Research – National 55.0%, Labour 29.6%, Greens 8.5% = Lead for National of 16.9%

So in the last term Colmar Brunton on average had the gap 1.7% tighter than Reid Research. Now again, that is not saying company A is correct and company B is incorrect. As I said these are polls generally taken over different time periods. But it is interesting there is some difference.

Curia’s monthly newsletter is out. The executive summary is:

Curia’s Polling Newsletter – Issue 66, March 2013

 March saw just three political polls published in New Zealand – a DigiPoll and two Roy Morgan polls.

 The average of the public polls has National 11% ahead of Labour – 3% less than in  February. The seat projection is centre-right 58 seats, centre-left 60.

 Australia has Labor and Gillard’s ratings seriously tanking.  The Coalition has a two party preferred lead of 16% to 18%, and is projected to win three times as many seats as Labor in the September election. Gillard has only 26% positive approval and 65% negative.

In the United States Barack Obama’s approval rating is has dropped significantly by 7%.

In the UK the Conservatives are still 11% behind Labour and the UK Independence Party is now polling at the same level as the Liberal Democrats.

In Canada the Conservatives are on 31%, NDP 28% and Liberals 27%. Likely Liberal Leader Justin Trudeau is already leading the Preferred PM polls.

The normal two tables are provided comparing the country direction sentiment and head of government approval sentiment for the five countries. New Zealand continues to top both by considerable margins.

We also carry details of polls in New Zealand on asset sales, constitutional issues, a four year term, Christchurch schools, same sex marriage, The Hobbit, cats, List MPs, and nuclear powered ships plus the normal business and consumer confidence polls.

This newsletter is normally only available by e-mail.  If you would like to receive future issues, please e- go to http://listserver.actrix.co.nz/cgi-bin/mailman/listinfo/polling-newsletter to subscribe yourself.

Another interesting UMR poll. I like the scale they used which was:

• Are absolutely certain it is true
• Are fairly certain it is true
• Believe it’s true but are not too certain
• Believe it’s true but are not at all certain
• Believe it’s not true but are not at all certain
• Believe it’s not true but are not too certain
• Are fairly certain it is not true
• Are absolutely certain it is not true

The first four can be totalled to be agreement and the last four disagreement, but as interesting as the totals are the breakdowns.

On belief in God, the responses were:

• 28% absolutely certain it is true
• 13% fairly certain it is true
• 9% believe it’s true but are not too certain
• 11% believe it’s true but are not at all certain
• 6% believe it’s not true but are not at all certain
• 5% believe it’s not true but are not too certain
• 11% are fairly certain it is not true
• 16% are absolutely certain it is not true

I am amused at the 44% who answer absolutely certain there is or is not a God. I’d love to know how they can be “absolutely” certain!

The demographic breakdown is interesting:

72% of women believe in God, compared with 52% of men.

That’s a large difference, and not one I was aware of previously. Why would 20% more women believe in God? Is it men are more into proof than faith?

65% of Maori and 72% of Pacific people believe in God.

Less surprising.

Another question was if you believed Jesus Christ was a real person who lived 2,000 years ago. This was regardless of if you think he was the son of God, a prophet, a preacher etc.

• 29% absolutely certain it is true
• 25% fairly certain it is true
• 10% believe it’s true but are not too certain
• 14% believe it’s true but are not at all certain
• 4% believe it’s not true but are not at all certain
• 4% believe it’s not true but are not too certain
• 6% are fairly certain it is not true
• 8% are absolutely certain it is not true

I’m surprised 14% are fairly or absolutely certain that Jesus never existed.

Gavin White at UMR blogs:

We asked New Zealanders which of the following statements came closest to their views on the origin and development of human beings: 

• A) Human beings have developed over millions of years from less advanced forms of life, but God guided this process 
• B) Human beings have developed over millions of years from less advanced forms of life, but God had no part in this process 
• C) God created human beings pretty much in their present form at one time within the last ten thousand years or so

To keep the write-up clear, I’ll refer to Statement A as ‘intelligent design’, Statement B​ as ‘pure evolution’ and Statement C as ‘creationism’.

I like that they didn’t just ask people do they believe in evolution or creationism but actually gave them statements to choose from. A good approach in my opinion, as different people may think creationism means different things.

The numbers show that, amongst New Zealanders:

• 26% believe in ‘intelligent design’
• 45% believe in ‘pure evolution’
• 23% believe in ‘creationism’
• 6% are unsure.

I’m surprised the level for “pure” creationism is so high, ie that one in four said they think humanity is less than 10,000 years old.

As you might expect, the American numbers are quite different:

• 32% of Americans believe in ‘intelligent design’
• 15% believe in ‘pure evolution’
• 46% believe in ‘creationism’
• 7% are unsure.

Only 15% of Americans believe in “pure evolution”. Oh dear. Now that I have a problem with people believing in intelligent design. There can be no proof for or against intelligent design. I can’t be certain there isn’t a creator who has been nudging things along.

However I can be pretty certain that humanity is more than 10,000 years old. Unless God spent a lot of time and effort in laying fake clues for us!

Paul Little writes in the HoS:

Family First got market research outfit Curia to survey attitudes on assaulting children – or “smacking” as they call it – as part of its perpetual campaign to have the law forbidding this overturned.

Whenever someone refers to smacking as assault, it isn’t hard to guess their views on the issue.

Family First appears to believe that not being smacked is the single most important issue facing our children. A “whopping” 77 per cent of those polled, FF claimed, wanted the law changed to bring back the bash.

Obviously, results on a question like this will depend on how it is framed. In this case, the 1000 respondents were asked whether the law should be changed so that “parents who give their children a smack that is reasonable and for the purpose of correction are not breaking the law”.

In other words, it was saying, “Do you believe something that is reasonable is reasonable?”

Critiquing of questions is always welcome. I do it myself to other polls, and it is important people understand the exact question asked.

In this case though what Mr Little overlooks is that the use of the term “reasonable” is not invented by Curia. It is the term that was in the law prior to 2007, and is the term used in the amendment that was put up by John Boscawen (and previously Chester Borrows), and is a term still in the law today. It is a term that was at the heart of the two year debate on the law.

“Should the law be changed so that it is legal to hit children when it is not legal to hit adults, or should there be one law for all?”

That would have got a different result. But it wouldn’t have been a very useful question to find out the answer to the question about whether people want the law to allow correctional smacking that uses reasonable force (as the law used to allow). I would also point out that the term reasonable force is still in the Crimes Act (s59) as parents can use reasonable force for:

• preventing or minimising harm to the child or another person
  
• preventing the child from engaging or continuing to engage in offensive or disruptive behaviour
• performing the normal daily tasks that are incidental to good care and parenting

So I think the use of “reasonable” is entirely reasonable considering the current law.

Two other questions in the survey were equally absurd. Seventy-seven per cent said the new law had had no effect on child abuse. All credit to Curia for finding 1000 people who were up to speed with the latest data in that field.

Mr Little seems to be suggesting that people can’t have opinions on an issue, unless they are an expert on the field. I presume he also decries polls that ask people if they think fracking is unsafe and should be banned?

It is absolutely valid to find out if people think a law change has worked. What isn’t valid is to take the results of the poll as meaning the law has not worked. This is about finding out what people think.

The official crime stats should that NZ has less violent crime than in the past. But I don’t think this means it is wrong to ask people whether they think NZ is safer than in the past.

I blogged today on a poll regarding safety of nuclear powered ships by UMR. Does Mr Little think UMR were wrong to ask non-experts on whether they think nuclear powered ships are safe?

For the problem with Question 3 – “Would you still smack your child if you thought it was reasonable … despite the current law?” – see Question 1, above.

Mr Little misses the point of this question. Apart from the fact that the term “reasonable” is used because it is in the former (and current) law, this question is about finding out if people will obey the law, even if they disagree with it. An overwhelming majority say they will ignore the law and do correctional smacking even though it is explicitly illegal. That is a quite extraordinary to have so many people say they will break the law.

UMR polled people in January on if they think nuclear powered ships are safe.

The answer should of course be yes, unless you regard every human activity known as unsafe.

A 1996 report found there had been no nuclear events from Western nuclear ships:

However, when it comes to  nuclear events the west has so far had a unique safety record  with no nuclear accidents.

They estimate a serious (but not nuclear) accident involving a nuclear powered ship every 50 years on average.

A more recent report says:

U.S. Nuclear Powered Warships (NPWs) have safely operated for more than 50 years without experiencing any reactor accident or any release of radioactivity that hurt human health or had an adverse effect on marine life. Naval reactors have an outstanding record of over 134 million miles safely steamed on nuclear power, and they have amassed over 5700 reactor-years of safe operation.

So the science is pretty clear on this one. But the poll result was:

• 38% think they are safe
• 48% not safe
• 14% unsure

That shows how ill-informed so many people are. But what I was most interested in was this:

59% of men now believe that nuclear powered vessels are safe, compared with only 18% of women.

That is a massive difference by gender. I can’t recall another issue on which there is such a difference.

If I had to guess, I’d say it is the fear of women that a nuclear accident could interfere with any pregnancies, as in lead to deformed babies. The maternal instinct over-riding risk analysis?

What other reasons do people think might explain the difference? More men study science?

Adam Bennett at NZ Herald reports:

Almost one in five New Zealanders who oppose the partial sale of Mighty River Power intend buying shares in the company anyway, according to a Herald-DigiPoll survey.

But the survey also indicates opposition to the sales plan is softening, with just over half of the 750 respondents saying they are against it compared with almost two thirds a year ago, and as much as three quarters before the 2011 election, which was largely fought on the issue.

According to the poll, conducted between March 11 and March 17 during the Government’s initial Mighty River Power advertising blitz, 52.2 per cent of respondents opposed the sale and 41.9 per cent supported it.

That’s a big change – from almost 75% against to just over 50%.

I have always assumed that any referendum vote would be a massive vote against.

Every CIR to date has always had a massive vote in favour of the desired outcome of the petitioners. The results have been:

• Firefighters 88%
• Size of Parliament 81%
• Justice reform 92%
• Smacking law 87%

How much of a political disaster would it be for Labour and the Greens if they lost the referendum? They’ve spent $400,000 of their parliamentary budgets on getting people to sign the petition. The referendum may cost the taxpayer up to $10 million to run. They’d be laughing stocks if they lost the vote they spent so much money on trying to achieve.

Of those opposed almost a fifth intended buying shares while 30 per cent of all those polled said they would buy shares. The survey has a margin of error of 3.6 per cent.

Turnout could be fascinating. Those who have purchased shares and support partial asset sales may be highly motivated to vote in the referendum as if the referendum endorses the sales it would discourage Labour, Greens and NZ First from confiscating their shares back after the election. And those against may wonder what is the point when one or two companies have already been floated by the time of the referendum.

I’ve blogged at Curiablog the latest poll results – a Roy Morgan poll and a Herald Digipoll.

Andrew blogs at Grumpollie:

The problem with calling cell phones doesn’t really lie in the cost of calls. For a polling company, calling a cell phone doesn’t cost that much more than calling a landline. The problem is the complexity and cost of employing dual sampling frames when the proportion of cell phone users without a landline is still very low. If the purpose of calling cell phones is to reduce non-coverage of likely voters, then you may actually need to ‘screen out’ those you call on cell phones who also have a landline (because they are already covered by the landline sample frame).

If we assume (hypothetically) that 6% of eligible voters have cell phones and no landline, that means that 94% of the people you call on a cell phone will not be eligible to take part (again, because they are already covered by the landline sample frame). This is where the cost would really begin to build up – all those interviewer hours required just to screen people out (eek!).

It is worth recalling the stat – you’d have to call almost 17 cell phones to find 1 person who doesn’t have a landline.

At the moment such a small proportion of New Zealanders have a cell phone with no landline that party support would need to be dramatically different among those people for this particular type of non-coverage to influence the poll results for party vote (eg, support for Labour among cell phone only voters may need to be twice what it is among landline voters for the party vote result to shift by more than, say, the margin of error).

Also a very useful point to recall when people raise the cellphone issue.

Let’s say National is at 48% amongst landline users and only 40% amongst cellphone only users. If you poll landline users only then you get National at 48%. If you poll both landline and cellphone only users you would have National at 47.5%. Not a huge difference.

Also worth noting almost all polling companies weight by age so any differences between landline and cellphone users which are due to different age profiles get compensated for anyway.

When the proportion of people with cell phones and no landline is considerably larger than it is today (like it is in some other countries), then it will definitely make sense to employ a dual sampling frame approach.

The results of the census will be interesting to see how the proportions have changed.

A group of New Zealand’s leading political pollsters, in consultation with other interested parties, have developed draft NZ Political Polling Guidelines.

The purpose is to ensure that AMRO and MRSNZ members conducting political polls, and media organisations publishing poll results, adhere to the highest “NZ appropriate” standards. The guidelines are draft and comments, questions and recommendations back to the working group are welcome.

This code seeks to document best practice guidelines for the conducting and reporting of political polls in New Zealand. It is proposed that the guidelines, once approved and accepted, will be binding on companies that are members of the Association of Market Research Organisations (AMRO) and on researchers that are members of the Market Research Society of New Zealand (MRSNZ).

The code only covers “political polls”, which for the purpose of the code are polls that related to public votes such as national elections, local body elections and parliamentary referenda. This is in recognition of the fact that reporting of polls may have an impact on how people vote.

For each issue, the code details, where appropriate: 

• Best practice for the market researcher conducting the poll
• Best practice for the market researcher in reporting results
• Best practice for the media in publishing and commenting on results

It is intended that the code assist politicians, political scientists, journalists and members of the public to be confident that political polls do in fact largely represent the opinions of the wider public and are a guide as to likely voting behaviours if an election were to be held at that time.

The development of the guidelines is in recognition of the fact that reporting of polls can have an impact on how people vote. Inaccurate polls, or polls that are interpreted and reported inaccurately, can impact on voting attitudes and behaviours and thus influence the democratic process.

It behoves all members of the polling and media communities to treat polling responsibly. Reliable polls (rather than informal surveys) require a high degree of rigour. These guidelines are designed to ensure that rigour is understood and applied.

The guidelines have been developed in the light of the European Society of Market Research (ESOMAR) international guidelines. ESOMAR is the global authority on research best practice and both AMRO and MRSNZ are ESOMAR members. It is recommended that those interested also read the ESOMAR /WAPOR guide to opinion polls and published surveys.

 NB: The term “must” indicates a requirement, while the term “should” indicates recommended best practice.

The guidelines also include a proposed easy reference guide for media:

1. If possible, get a copy of the full poll report and do not rely on a media release.
2. The story should include the name of the company which conducted the poll, and the client the poll was done for, and the dates it was done.
3. The story should include, or make available, the sample size, sampling method, population sampled, if the sample is weighted, the maximum margin of error and the level of undecided voters.
4. If you think any questions may have impacted the answers to the principal voting behaviour question, mention this in the story.
5. Avoid reporting breakdown results from very small samples as they are unreliable.
6. Try to focus on statistically significant changes, which may not just be from the last poll, but over a number of polls.
7. Avoid the phrase “This party is below the margin of error” as results for low polling parties have a smaller margin of error than for higher polling parties.
8. It can be useful to report on what the electoral results of a poll would be, in terms of likely parliamentary blocs, as the highest polling party will not necessarily be the Government.
9. In your online story, include a link to the full poll results provided by the polling company or state when, and where, the report and methodology will be made available.
10. Only use the term “poll” for scientific polls done in accordance with market research industry approved guidelines, and use “survey” for self-selecting surveys such as text or website surveys.

It is important to stress that these guidelines are a draft only, and the pollsters are keen to get feedback from interested people. Specifically suggestions for possible other requirements, amendments to proposed guidelines or even removal of a specific point.

Please send any comments, questions and submissions on the guidelines to pollingguidelinesgroup@amronz.org.nz.

The date listed as the closing date of 1 March is incorrect. Feedback is still welcome, and it is planned in May to hold public consultation meetings in Auckland and Wellington.

Note that I was a member of the working group that developed the guidelines, as were representatives from UMR, Ipsos, Digipoll, Reid Research and Colmar Brunton and the MRSRZ President and AMRO Executive Director.

Gavin White from UMR blogs:

Some of you may have seen some of our research commented on in the media earlier in the week. The research has been interpreted as supporting Gareth Morgan’s campaign on cats, but I don’t think it’s anywhere near as clear cut as that. 

In other words, don’t believe the spin of a campaign that selectively quoted the results.

The media reports focussed on one statistic: the fact that 54% of New Zealanders supported some form of controls that would reduce the future population of cats, once told that an Otago University study estimates 1.12 million native birds are killed by domestic cats each year in New Zealand. The question cannot, however, be treated in isolation: the other questions in the survey make clear that the sorts of controls people are actually prepared to have are actually pretty mild.

So what were the full results.

• 62% believe that all cats should be neutered or spayed.
• 57% think that cats should be banned from areas near wildlife reserves, forests and national parks
• 53% believe that all cats should be registered and microchipped
• 42% consider that all cats should wear bells
• Just 12% believe that cat owners should not replace their cats when they die
• Only 7% think that cats should be kept indoors at all times of the day.

This reflects I think the common sense approach of New Zealanders. Only 7% agree with the more extreme proposals from Morgan, but a bit over half agree with some of the more moderate stuff.

Gavin also points out:

In the question on banning cats from near wildlife reserves, forests and national parks, I suspect that many people would have used a reasonably narrow definition of the word ‘near’ (e.g. within a few streets of the park boundary).  It would be stretching the case to say that the poll supports banning cats from whole suburbs or towns (like Karori, which is adjacent to the Zealandia wildlife reserve, or Ohakune, which is near the boundary of Tongariro National Park).

A very good point also. It’s good to have pollsters commenting on their own research publicly – they are often the one best placed to know what limits there are in interpreting what it means.

Matthew Backhouse at the Herald reports:

The philanthropist’s Cats To Go website was initially met with outrage from cat lovers, but responses to questions in UMR Research’s monthly online survey show the public may be coming around to his views.

Not really. The survey showed that the moderate proposals have support – as they should. But there is total rejection for his more extreme proposals.

The survey of about 1000 people found more than half supported neutering all cats, registering and microchipping all cats, and banning cats from areas near wildlife reserves, forests and national parks.

This is not surprising. And if Morgan had campaigned on those issues solely, then I’d say he would have got a far better reception.

But two of the measures Dr Morgan advocates were met with far less support, with only 12 per cent agreeing cat owners should not replace them when they die and 7 per cent agreeing cats should be kept indoors at all times.

And these measures were at the core of his campaign. His website is called Cats To Go – not Cats To Be Registered.  The top infographic on his site says “Make this cat your last”.

Far from endorsing his views, this poll is a total rejection of his extremism.

The graph above shows the average of all the public polls since the 2011 election.  Have just published Curia’s monthly newsletter:

February saw just five political polls published in New Zealand – one each from Colmar Brunton, Reid Research and Ipsos and two Roy Morgan polls.

The average of the public polls has National 14% ahead of Labour – the same as January. The seat projection is centre-right 61 seats, centre-left 57.

Australia has Labour and Gillard in trouble. The Coalition lead by 9% to 10% and for the first time Tony Abbott has overtaken Julia Gillard as Preferred Prime Minister.

In the United States Barack Obama’s approval rating is at +8%, and the country direction is -23%. The generic congressional poll average has the Democrats 3% ahead of the Republicans.

In the UK the Conservatives are now 11% behind Labour. They came third in the recent Eastleigh by-election. The Lib Dems retained the seat but the UK Independence Party came second.

In Canada the Conservatives are on 32%, NDP 28% and Liberals 25%.

The normal two tables are provided comparing the country direction sentiment and head of government approval sentiment for the five countries. New Zealand continues to top both by considerable margins.

We also carry details of polls in New Zealand on Hekia Parata, asset sales, four year term, Christchurch jobs, David Bain, same sex marriage, adoption plus the normal business and consumer confidence polls.

The full newsletter is only available by e-mail.  If you would like to receive future issues, please go to http://listserver.actrix.co.nz/cgi-bin/mailman/listinfo/polling-newsletter to subscribe yourself.

I’ve just updated at Curiablog the full poll results from the One News Colmar Brunton poll and one of the issues they polled on was a four year term. The question they asked was:

“The current term of Government in New Zealand is three years, after which a general election is held.
Recently some politicians have suggested increasing the term from three to four years. Do you support
increasing the term of Government to four years?”

The results were:

• 56% yes
• 40% no

That is a welcome sign that a referendum in 2014 on the issue could succeed. I think it would be important that any change would not come into effect immediately but after the 2017 (or even 2020) election. That way it is not a poll on an extra year for the current Government.

The Parliamentary Library has also just published a research paper on parliamentary terms.

I’ve blogged the latest Roy Morgan poll at Curiablog.

The centre-left have dropped 6%.

I’ve blogged at Curiablog the results of the latest 3 News Reid Research poll.

Note that the poll was done after the release of the Auditor-General report into the proposed Sky City convention centre.

James Dann is a Christchurch local body politician and blogs on the Christchurch rebuild etc. His latest blog post is on the Nielsen survey for CERA which has upset a few people because in fact it found not everyone was unhappy.

James has said the survey is unrepresentative:

The key to this survey is in the methodology. It was undertaken between August 29 and October 15 last year. 2381 people responded to the survey. 1156 of them were from Christchurch City, 618 from Selwyn District and 607 from Waimakariri District. So that means that of the people who did respond, more than half (1225) AREN’T ACTUALLY FROM CHRISTCHURCH. I mean no disrespect to the people of Rolleston, Lincoln, Rangiora, Kaiapoi etc. I am sure they have had a hard time. But they don’t have the same issues that people who actually live in the city do. They haven’t had to deal with a lack of services, portaloos, red zoning, TC3 land, roadworks to the same extent that people who live in the city do. …

Yet the survey then merges these three population groups, and continues to extrapolate from the combined population for another 100 pages. If you get to the end of the report, you’ll find the populations of Christchurch, Selwyn and Waimakariri. They are 289,000, 30,000 and 36,000 respectively. More than 80% of the adult population lives in Christchurch, and yet more than 50% of respondents come from less than 20% of the population, a section of the population that the survey shows have a more positive response in this survey.

Lies, damn lies, and statistics.

Now the Herald has gone and reprinted his blog post in the Herald.

James has made a fatal mistake. He has not read the survey results.

He is correct that the raw sample is not reflective of the overall population. That 80% live in Christchurch but only half the responses were from Christchurch people.

But what he has either deliberately or accidentally omitted, is that the results are weighted so that the results do precisely match the area, gender and age of the local population. Nielsen say  in their report:

The sample design over-sampled residents of the two districts with smaller populations to ensure that the sample size within each district was sufficient to allow reliable and robust analysis. 

At the analysis stage, the data was adjusted by a process called weighting. This process adjusts for discrepancies between the profile of people who completed the survey and the known profile of residents of greater Christchurch. 

Population statistics are obtained from Statistics New Zealand data and is based on the latest population projections. 

Weighting increases the influence of some observations and reduces the influence of others. So, for example, while 618 or 26% of completed interviews came from Selwyn District, the population of Selwyn actually represents about 8% of greater Christchurch. Thus, the data was adjusted so that 8% of any ‘greater Christchurch’ result reported is based on the responses of Selwyn residents.

So the criticism of Dann are entirely invalid, and worse the Herald has repeated them on their website. This is not a matter of opinion. His failure to mention that the results are weighted by area totally invalidate his criticism. The Herald was aware of this also, as one of their editorial staff tweeted his blog post and I pointed out this failing in a response (which they retweeted).

Appendix 5 of the poll results on page 103 details their weighting matrix with precise quotas for area, gender and age. It is a model of transparency.

Incidentally how significant were the differences in responses from those in Christchurch itself, and the two neighbouring districts? Here’s the figures for overall quality of life – good vs poor.

• Christchurch City 72% good vs 7% poor
• Selwyn 85% good vs 4% poor
• Waimakariri 82% good vs 4% poor
• Greater Christchurch 74% good vs 7% poor

So the results for Greater Christchurch are almost identical to Christchurch City.

The only question I have is whether Mr Dann knew the survey was weighted and decided not to mention that fact, or whether he just failed to read the results in full.

UPDATE: He has amended his original blog post to now mention the weighting, but note that the version picked up by the Herald doesn’t include this.

UPDATE 2: Dann was a Labour Party candidate at the last local body elections. Yet the Herald didn’t think to mention this. Shocking.

The latest poll (Fairfax Ipsos) is at  Curiablog. It shows no Government could be formed without the Maori Party.

I’ve blogged at Curiablog the details of the One News Colmar Brunton poll that was released this evening.

Just published the monthly polling newsletter. The summary is:

January saw just two polls published – both Roy Morgan polls.

 The average of the public polls has National 14% ahead of Labour in January, 2% greater than in December. The seat projection is centre-right 57 seats, centre-left 61.

 Australia has divergent poll results with Roy Morgan having the Coalition 3% ahead and Newspoll 12% ahead. Newspoll also has Gillard’s approval rating dropping and Abbott’s rising.

 In the United States Barack Obama’s approval rating is at +10%, and the country direction is -22%. The generic congressional poll average has the Democrats 5% ahead of the Republicans.

 In the UK the Conservatives are 8% behind Labour, having gone up in the polls after announcing a future referendum on membership of the EU.

 In Canada the Conservatives are on 25%, NDP 30% and Liberals 22%.

 The normal two tables are provided comparing the country direction sentiment and head of government approval sentiment for the five countries. New Zealand continues to top both.

 We also carry details of polls in New Zealand on Maori voters, Prince Charles, same sex marriage, euthanasia, housing affordability, The Hobbit, drink driving plus the normal business and consumer confidence polls.

The full  newsletter is only available by e-mail.  If you would like to receive future issues, go to http://listserver.actrix.co.nz/cgi-bin/mailman/listinfo/polling-newsletter to subscribe yourself.