The importance of questions
Almost nothing is more important in polling than setting the questions. And two recent reports of polls reminds me of this:
The first on cellphone use is reported as:
A new Research New Zealand (RNZ) poll showed 86 per cent of New Zealanders believed that phoning/texting and driving should be illegal.
Now they don’t state the actual question asked, but my first reaction is that by including texting and phoning together could lead to very different results to asking separately. Also talking about “phoning” rather than “using a phone” could get different results as one conjures up dialling out on a phone, rather than merely answering a call. I think texting someone while driving is pretty reckless, but answering a phone call is far less of a distraction.
A second is on Easter Trading:
Research New Zealand surveyed 500 people to find out whether they thought shops should be open on Good Friday and Easter Sunday.
Director Emanuel Kalafatelis says only 32 per cent of those polled want the law to change.
Now if you asked “Should shops have the option of opening” as opposed to “Should shops be open”, you may get different results. For example I don’t think shops should open on ANZAC Day morning, but I don’t think the law should prohibit them.
Now I am not saying in either of these cases, that a differently worded question would change the majority support into majority opposition. But that the questions, as reported, may make people more likely to respond a certain way.
This is not a criticism of Research New Zealand, a very good company. Apart from anything else I have not seen the actual questions asked – only a media summary. The reason for the post is to highlight how important the wording of questions are, and how media reports which do not include the actual question used need to be treated with some caution.
March 22nd, 2008 at 11:21 am
Isn’t this carefully loaded proposition then use of teh ‘evidence’ it evinces how much of our latest policies have been formulated? Helen herself is a master at this.
How many people would say ‘yes’ to the question: ‘Should a Trade Union be obliged to register with the government at the beginning of each year before an election is called?’
March 22nd, 2008 at 11:23 am
“The reason for the post is to highlight how important the wording of questions are, and how media reports which do not include the actual question used need to be treated with some caution.”
I find almost all poll questions to be extremely poorly formulated and designed to favour a certain outcome. Why don’t poll companies provide more objective questions?? When their questions are so consistently badly formed, it only opens the door to allegations of political cronyism. Perhaps they should offer for each question a pro and an anti scenario.
For example-
Question 1- Do you believe the use of cellphones and texting whilst driving should be banned?
Question 2 (alternative) – Whilst crime is rife, violence is on the upswing, police are unable to provide even basic levels of civil protection and cannot respond in a timely fashion to emergency situations, should they be messed about by the introduction of even more useless unenforceable fucked in the head socialist style legislation???
March 22nd, 2008 at 11:25 am
For example;
Question 3 – why do Maori’s have the highest imprisonment rate in the world?
Question 4 – why are child abuse statistics appalling ?
March 22nd, 2008 at 1:08 pm
Do you think it’s a good thing that our current cannabis laws enable people in organised gangs to make hundred of thousands of dollars a year selling something that is no more expensive to grow than tomatoes?
Do you think a law that makes something as cheap to grow as tomatoes worth $8,000 a kilo is a sensible law?
Would you grow tomatoes if you could sell them for $8,000/kilo?
March 22nd, 2008 at 1:21 pm
Oh the irony. I was just thinking earlier how loaded the question in DPF’s own poll was. Currently it is: “Should employees be allowed to work on a public holiday if they wish to earn extra money?” Talk about influencing the outcome. Pot, Kettle, Black.
[DPF: Good God - that poll is obviously taking the piss and not pretending to be a scientific poll]
March 22nd, 2008 at 1:56 pm
Sean
That’s the whole point. It’s good that you get it.
March 22nd, 2008 at 2:17 pm
burp – …yeeeahhh, of course…..
March 22nd, 2008 at 2:59 pm
Yes, the phrasing of the question is very important and it will influence the repsonse. I think any social scientist would agree.
The NZ Herald’s regular questions to readers are interesting in this regard. They seem to be designed not to genuinely survey public opinion but to raise people’s blood pressure.
March 22nd, 2008 at 3:24 pm
[DPF: Good God - that poll is obviously taking the piss and not pretending to be a scientific poll]
Obviously, especially with the answer options. Though in the context of your mood yesterday I wouldn’t be so sure it was just about “taking the piss”. A new day is upon us though
March 22nd, 2008 at 6:49 pm
For once I agree with Redbaiter. People talking on cellphones was the last thing on my mind a few weeks ago as I narrowly avioded one bunch of kids beating the living daylights out of another bunch of kids in the early evening in central chirstchurch. The police told me far worse happens in board daylight in the central city. As for the recent boy racer public road block, riot and roadside petrol burning… well not a single one of them was arrested- why? Police didn’t have the man power that night so the boy racers got away with it. I’ve since enrolled in a self defence class, its very crowded too. Funny that.
March 22nd, 2008 at 8:02 pm
Interesting stuff.
The best research seems to show that overall cellphone related road crashes are a very very small proportion of the total (there will be under-reporting though).
Even so the glorious NZ media has embarked on the Liabore Party campaign to ban them. This is so Dear Leader can be seen to be “doing something”. This is much easier than asking the police to enforce road rules rather than sitting eating doughnuts in speed camera vehicles.
To me, it seems that as long as we keep campaigning against cellphones rather than assessing the evidence surrounding them then they must be eventually be banned for political reasons.
Even so. Texting on the move is surely plain dumb.
March 22nd, 2008 at 11:49 pm
I don’t agree texting is necessarily worse than talking on a cell phone because you can text a letter at a time as the conditions permit.
It is the thinking process that distracts. Did no one see the test on one of the driving programs where they asked a racing driver to negotiate some cones while answering some maths questions? He went from a perfect result to knocking over nearly every cone! That is why tests have shown handsfree phones to be nearly as much of a distraction as handheld phones.
Maybe a compromise would be to ban using a phone while driving for more than one minute. I saw a woman recently starting driving her car in Johnsonville, immediately start talking on her cell phone and then proceed to make her way onto the motorway and drive all the way down the Ngaranga Gorge, talking the whole way and driving relatively slowly because she was concentrating on her conversation! I was stuck behind her and there wasn’t an opportunity to safely pass her.