A poll of 47

November 8th, 2011 at 1:00 pm by David Farrar

The Herald each election does a Mood of the Nation survey when they get reporters onto the street and interview around 500 people. I think this generally is a good initiative. They report that it is unscientific, but it allows them to use actual quotes from people to use in their stories. So long as they stress the survey is unscientific, and the focus is more on what people said rather than the numerical results, I think that is fine.

But you can take these street surveys too far. Off memory, in 2008 a street survey of 100 people was reported by the Herald with great prominence as showing Judith Tizard retaining her seat. She lost of course.

This time we have a street survey of just 47 people in , that generates a story. It says:

John Banks has some support in the wealthy suburb of Remuera, but is less popular on the liberal fringes of the Epsom electorate, according to a Herald street survey.

A poll of 47 Epsom voters yesterday found the National candidate ahead of Act’s Mr Banks by 22 votes to 20.

It is ridiculous to do a story on a street poll of 47. First of all, you have the sampling problem – that in fact it is a poll of people who happened to be out on a street – in no way random.

But even if you overlook the fact it is a street poll, the sample size is ridiculously low.  The margin of error is 14.7%!

I generally regard 300 as the minimum acceptable for an electorate poll. That gives a 5.8% margin of error. A sample of 47 is close to useless.

Tags: , ,

11 Responses to “A poll of 47”

  1. Richard29 (377 comments) says:

    I like MMP and will support it in the referendum but the fact that they have a Green party voter saying they’ll give their electorate vote to National is a good illustration of the main thing that needs fixing with MMP.

    Parties that get less than the threshold should not get an exemption just because of an electorate, it encourages this kind of tactical voting. I resent the fact that the voters of Ohariu, Epsom, Rodney wield disproportionate power over the election outcome.

    This is one of the main reasons for avoiding a First Past the Post system, you end up with 80% of the electorates in the country being a foregone conclusion and politicians lavishing attention (and money) on the handful of ‘swing’ electorates.

    The end result is that lots of people just don’t even bother voting anymore. The most recent congressional election in the United States had a 45% turnout.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  2. Adolf Fiinkensein (2,875 comments) says:

    Don’t they have form for this sort of nonsense? Was this the same Herald which screwed ACT a few years ago with a highly dubious and damaging poll published only a day before the election?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  3. tristanb (1,133 comments) says:

    Green party voter saying they’ll give their electorate vote to National is a good illustration of the main thing that needs fixing with MMP.

    What’s wrong with that?

    It’s no more wrong than it would be to encourage National supporters to vote ACT. (Or Joker card Dunne)

    It’s not really the problem with MMP. If I was a Green voter and this was FPP, I’d still vote for the National MP just to keep Banks out.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  4. tristanb (1,133 comments) says:

    Argh, can we not edit anymore?

    Anyway, I agree that the threshold thing needs to be looked at. But a vote for Banks will only bring Banks in. And, even as a potential ACT supporter, I would not want this.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  5. marynicolehicks (24 comments) says:

    I disagree. You can have a poll with a sample of less than 300 for an electorate poll. The results just have to have a very significant difference that is in the order of magnitude larger than the margin of error. An good example is the gap between National and Labour.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  6. ben (2,418 comments) says:

    The view seems to be that announcing it is unscientific gives them a free pass to use whatever methodology they like.

    But think about what the ‘unscientific’ disclaimer means: it literally means that there is no way to know if its results are true, because the method to detect this, science, has been abandoned in producing the numbers.

    And if the Herald staff happen to be interested in the result, then all kinds of shenanigans could be going on. If they systematically avoid men wearing suits or women pushing prams, or sample from 7am-9am only, or 3pm-4pm only, for example, they can skew results any way they want without any requirement beyond its disclaimer to disclose. And that’s assuming they didn’t make up the numbers entirely. Unscientific indeed.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  7. East Wellington Superhero (1,151 comments) says:

    A poll in an Auckland high-rise office of 47 would probably got 65% Nat, 20% Lab, 10% ACT, 5% Green.
    Or perhaps we could poll the 47 dock workers at Ports of Auckland. 80% Lab.
    Or stand outside the Auckland Uni student association. 60% Lab, 30% Green.
    Or at a Grey Lynn café at 2pm 90% Green.

    Folly.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  8. Scott Chris (6,017 comments) says:

    They probably did do a poll of 500. The first 47 polled at 22 to 20 in favour of Goldsmith. The last 453 gave a result revealed 37% support for Goldsmith, 23% for Banks and the rest shared twixt the minor parties and undecideds.

    But there’s no story in a walkover…. or maybe the Herald’s editor is an ACT supporter.

    Why else would you report such meaningless numbers?

    Smells just like the Colin Craig Conservative Party bent poll.

    Poor Banksie’s gettin’ desperate.

    Don’t suppose Cameron Slater wants to look into this one.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  9. side show bob (3,660 comments) says:

    Why do they bother polling at all?. Why not just poll the Herald reporters, after all it’s their mood that dictate how they would want us to vote.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  10. leftyliberal (642 comments) says:

    Another thing to remember is that the margin of error in and of itself should not be used as a measure of the error in a poll – reason is that the two main runners are typically negatively correlated (eg National goes down -> Labour goes up). A good rule of thumb is that you want twice the difference of the maximal margin of error. For a sample of 47 that’s 2/sqrt(47).

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  11. Mark (1,471 comments) says:

    Richard29 I agree with the sentiment of your comment that parties should achieve at least 5% before they are eligible for a seat.

    There would be a delicious irony if the Green and labour party voters gave their electorate vote to the national Party Candidate. The saving grace for National and ACT is that it hard to imagine that there are many Labour Party or Green Party voters in Epsom.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote

Leave a Reply

You must be logged in to post a comment.