Cell-phone polling

October 30th, 2012 at 10:00 am by David Farrar

Dim-Post highlights a comment left on his blog an an anonymous pollster:

The problem with calling cell phones doesn’t really lie in the cost of calls. For a company, calling a cell phone doesn’t cost that much more than calling a landline. The problem is the complexity and cost of employing dual sampling frames when the proportion of cell phone users without a landline is still very low. If the purpose of calling cell phones is to reduce non-coverage of likely voters, then you may actually need to ‘screen out’ those you call on cell phones who also have a landline (because they are already covered by the landline sample frame).

If we assume 6% of eligible voters have cell phones and no landline, that means that 94% of the people you call on a cell phone will not be eligible to take part (again, because they are already covered by the landline sample frame). This is where the cost would really begin to build up – all those interviewer hours required just to screen people out (eek!).

This is not the only way to reduce non-coverage – but it’s actually one of the more straight forward and ‘statistically pure’ ways (ie, you can develop some sort of weighting scheme, but the more you weight, the greater the design effect (which increases the margin or error, and decreased the accuracy of a poll).

To make things more complex:

– Some people have more than one cellphone, meaning that the probability of them being called is higher, so additional weighting would need to be applied to adjust for the probability of selection (you may notice that some polls weight by household size and the number of landlines connected to a house – this is adjusting for the probability of section)

– There are a lot of cell phone numbers that are out of use, but when they are called they still go through to a voice mail. Unlike landlines (which you can ‘ping’ to test the connection), it is very difficult (ie, near impossible) to determine if there is actually an eligible person at the end of a number, so you’ve got no measure of the success rate of your sampling approach (ie, refusal rates, response rates, qualifier rates etc).

At the moment such a small proportion of New Zealanders have a cell phone with no landline that party support would need to be DRAMATICALLY different among those people for this particular type of non-coverage to influence the poll results for party vote (eg, support for Labour among cell phone only voters may need to be TWICE what it is among landline voters for the party vote result to shift by more than, say, the margin of error).

When the proportion of people with cell phones and no landline is considerably larger than it is today (like it is in some other countries), then it will definitely make sense to employ a dual sampling frame approach. In NZ though (at least in 2011) most pollsters got things pretty close to the election day result so this would suggest non-coverage of cell phone only voters isn’t a big issue just yet. If cell phone plans get cheaper, then polling approaches will probably need to change to keep up.

All comments I agree with. As time goes on it will become more of an issue in NZ, but at this stage the impact is limited. Also polling companies compensate for those without landlines by having quotas or weighting by age and income.

6 Responses to “Cell-phone polling”

  1. Rick Rowling (892 comments) says:

    Yes, but one can’t miss any opportunity to smugly say “but who still has a land-line these days? I’m only on broadband, cell and skype of course.

    Similar to the “I don’t even have a TV in my house” bores.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  2. gazzmaniac (2,842 comments) says:

    I’m quite happy with that policy.
    It means that I don’t get annoyed by pollsters at dinner time, since I have no land line.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  3. Spoon (105 comments) says:

    Do you (and other pollsters) follow a do-not-call register DPF? Surely any such register would be weighted towards National voters (at a guess).

    [DPF: All members of AMRO and MRSNZ follow the MRSNZ do not call register. Very few numbers are on it, compared to the Marketing Association one]

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  4. annie (540 comments) says:

    Anything that slows the bastards down is fine by me. The less credible their results, the more likely they will go out of business and leave me alone.

    Do-not-call register? Don’t make me laugh. Been on it since its inception, still on it, makes not a blind bit of difference.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  5. leftyliberal (655 comments) says:

    @DPF: are VOIP lines included as “land lines”? With more and more switching to naked broadband (and it’ll become a heap more once fibre uptake starts kicking in) this could become significant.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  6. mikenmild (23,645 comments) says:

    I seem to recall the eager ACToids back in the day banging on about their low poll numbers being due to pollsters not calling their supporters’ cellphones and those same supporters being out and about a lot and too energetic, creative and hard working to be lolling around at home answering the phone.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote