The role of evidence in policy formation and implementation

September 9th, 2013 at 10:00 am by David Farrar

A very good report on the role of evidence in policy formation and implementation from Sir . I am a fan of more use of evidence based policy.

Sir Peter notes:

All of this occurs within a complex and uncertain environment where human responses and decision-making are influenced by many factors other than well-informed knowledge. Indeed, as I have stated previously, my view is that quality evidence should be seen as base knowledge on which, in a democracy, multiple values and associated perspectives must be overlaid. However, where evidence is conflated with values, its power is diminished. Where evidence is not considered properly, the risk of less than desirable policy outcomes is inevitable.

For instance, it is possible for the research process to be corrupted by inputs that are not objective, or by the failure to recognise personal biases in bringing forward evidence. Researchers can become impassioned advocates for a cause that their expertise could meaningfully inform dispassionately

I can think of a number of areas, such as public health, where researchers are impassioned advocates to put it mildly.

They key recommendations are:

  1. Develop a standard set of protocols across government regarding obtaining expert scientific advice;
  2. Extend the use of Departmental  Advisors (DSAs) more broadly across government;
  3. Use the community of DSAs and the Chief Science Advisor to assist central agencies with longer-term planning, risk assessment and evaluation;
  4. Improve and make more explicit the use of government funds for research to assist policy formation;
  5. Provide greater transparency regarding the use of research-informed data (or its absence) with respect to complex and controversial areas of decision-making where the public is directly or indirectly consulted.

I like the idea of each principal agency having a Departmental Science Advisor.

Tags: ,

33 Responses to “The role of evidence in policy formation and implementation”

  1. kowtow (7,932 comments) says:

    No more new tax payer funded jobsworths.

    Common sense used to be the criteria.

    Now it would be a question of which ideology a particular scientist followed.

    Just look at the mess we have with so called science based AGW.

    Vote: Thumb up 6 Thumb down 0 You need to be logged in to vote
  2. JeffW (324 comments) says:

    Why use evidence when left wing dogma is available?

    Vote: Thumb up 8 Thumb down 0 You need to be logged in to vote
  3. Redbaiter (8,009 comments) says:

    The PC appeaser Gluckman prefers the comfort of Conventional Wisdom to real intellectual scrutiny and Key should fire his useless arse.

    Vote: Thumb up 9 Thumb down 2 You need to be logged in to vote
  4. Griff (6,990 comments) says:

    http://sciblogs.co.nz/hot-topic/2013/08/13/gluckman-gets-it-wrong-being-alarmed-is-not-alarmist/

    Laidlaw: The report is remarkably restrained …you say in the introduction that the most probable future scenarios are cause for concern. But unless there’s some sort of political miracle we’re going to be looking at an average temperature increase of somewhere between 3.6 and 5.3 degrees centigrade over this century. This is rather more than cause for concern I would have thought. It’s going to be a catastrophe.

    Gluckman: I would agree personally.

    Vote: Thumb up 1 Thumb down 4 You need to be logged in to vote
  5. flipper (3,838 comments) says:

    So does the Science Advisor get to report to the Minister direct, or via the CEO?

    To be effective, the Science Advisor’s report would need to be published in full and attached to any departmental report/recommendation.

    Otherwise…same old, some old…. ?

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  6. tvb (4,247 comments) says:

    The Greens will find a way to criticise this as they make policy based on prejudice. Labour-Green governments are proving to be highly toxic and hugely unpopular in Australia.

    Vote: Thumb up 5 Thumb down 0 You need to be logged in to vote
  7. Bill Courtney (160 comments) says:

    DPF: ” I am a fan of more use of evidence based policy.”

    Really? How about “Where is the Isaac Report?” which should have been produced at the end of the Catherine Isaac led charter school Working Group. Where is the report that should have set out what works, what doesn’t and why, before NZ embarked on this ideological nonsense?

    How about other education favourites, such as performance based pay for teachers? The OECD has stated many times that there is no empirical evidence at all that this produces better outcomes for students, but still the ideologues pursue it.

    And the class size debacle itself last year still rates as the worst piece of “gut feel” policy making that was never subjected to rational analysis of any sort.

    What a load of hypocrites you lot are!

    Vote: Thumb up 3 Thumb down 9 You need to be logged in to vote
  8. kowtow (7,932 comments) says:

    The “science” is far from settled and there are honest scientists out there but they get little air time from a progressive media or politicians.

    http://opinion.financialpost.com/2013/08/22/lawrence-solomon-model-mockery/

    Vote: Thumb up 5 Thumb down 0 You need to be logged in to vote
  9. Redbaiter (8,009 comments) says:

    No evidence performance based pay works.

    Yeah right.

    Teacher’s Unionist speaking you reckon?

    Vote: Thumb up 9 Thumb down 1 You need to be logged in to vote
  10. Pongo (371 comments) says:

    http://www.dailymail.co.uk/news/article-2415191/Global-cooling-Arctic-ice-caps-grows-60-global-warming-predictions.html

    Wonder whatGluckman makes of this, he wants things evidence based well 17 years and no warming compared with supercomputing models that so far have got everything wrong. Anyway don’t expect this story to be repeated anywhere in the MSM despite their usual running off to Salinger after any mild weather event.

    Vote: Thumb up 4 Thumb down 1 You need to be logged in to vote
  11. backster (2,122 comments) says:

    More bureaucracy, more non productive highly paid politically correct boffins, less commonsense. greater ability to delegate responsibility and avoid blame, seems to be whats recommended.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  12. Nigel Kearney (915 comments) says:

    Bill Courtney, above, misapplying an evidence based process to suit his own biases is a good example of the risks.

    I am skeptical about whether performance pay actually lifts individual performance very much. But it definitely makes it more attractive for high peformers to stay and low performers to leave, which increases the peformance of the organisation as a whole.

    Vote: Thumb up 6 Thumb down 0 You need to be logged in to vote
  13. flipper (3,838 comments) says:

    Bill Courtney (81) Says:
    September 9th, 2013 at 10:40 am
    ****

    Humbug.
    F*** the tax free OECD salaried mob, or the tax free UN mob.

    Look at the UK and the USA…education systems with the union anchors throttling them.

    Of course, you will have voted against Rudd (inferentially Gillard) on Saturday, because the Aussie system, has told their education unions to get stuffed. Was it Abbott? No it was Gillard, under Rudd.

    Time that you went back to school – preferably a private or a partnership/integrated school. And BTW, no interest free student oans. Pay your own way, including interest..

    Vote: Thumb up 2 Thumb down 0 You need to be logged in to vote
  14. Nostalgia-NZ (5,039 comments) says:

    ‘However, where evidence is conflated with values, its power is diminished.’

    That’s correct, but will a scientist manage to keep conflicting political points out, I don’t think so because they won’t be the final arbiters. New Laws for example are always going to be juggled to get the votes needed to pass them, the ‘look’ for the public being the most desirable outcome – potential for a flawed outcome whichever way you look at it even if beginning from an evidence based foundation.

    Vote: Thumb up 2 Thumb down 0 You need to be logged in to vote
  15. Griff (6,990 comments) says:

    :lol:
    2013 – 17 + 1996
    Trend: 0.110 °C/decade +- 0.120
    http://www.skepticalscience.com/trend.php
    Idiot
    Statistical significant warming is apparent if you use enough data points.
    Using a cherry picked start date then spin bullshit is the actions of a statistically illiterate moron.
    As is pointing out that the ice extent is up 60% yet failing to quantify this in the historic record.
    Up 60% from last year. The extent is the same as 2009 was
    Down From the 1980 to 2010 average by 1,200,000sqkm
    http://nsidc.org/arcticseaicenews/charctic-interactive-sea-ice-graph/

    Vote: Thumb up 2 Thumb down 5 You need to be logged in to vote
  16. peterwn (3,211 comments) says:

    The former New Zealand Electricity Department had a ‘research engineer’ which was just like a scientist advisor. One studied the potential effects of earthquakes on the electricity supply system – he became a world authority on the subject. Thanks to his work, the national grid and grid exit substations in Christchurch survived the earthquakes with minor interruptions only. His successor (Keith Turner later CEO of Meridian Energy) was asked to assess the liklihood of salt spray pollution for a proposed coastal substation. He hit upon the idea of analysing grose bushes for salt content which gives a good indicator of the amount of salt spray. In both cases, principled objective decisions could be made.

    However some government agencies and university departments are not up to the task. Peter Gluckman quoted the lack of scientific studies on the proposal to increase class sizes. IMO there was no way the responsible Minister was going to receive principled advice on this from education officials or be able to rely on academic research from university education departments. Similarly Crusher Collins is hardly likely to take heed of views of university criminology departments when they are full of crim huggers.

    Another issue – some issues are highly political and no amount of science may change this. For the example a government is not going to make water supply fluordation or childrens’ vaccinations mandatory despite overwhelming scientific evidence supporting both.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  17. Albert_Ross (270 comments) says:

    Peterwn, the scientific evidence does not support making water supply fluoridation mandatory. The scientific evidence supports the view that putting fluoride in the public water supply helps make people’s teeth healthier. That is a scientific statement – one that can be tested by scientific methods, experimentation and empirical experience

    However, “the Government should therefore make it mandatory to fluoridise the publc water supply” is not a scientific statement. It is a political one, requiring a view about the extent of the Government’s right and responsibility to incur public expense in order to improve private health outcomes. If the Government has such a responsibility, why does it stop short of mandatory daily exerise classes and force-feeding people with fresh vegetables?

    Decisions should most definitely be informed by a clear understanding of the relevant science, but the science alone will never be enough to drive the policy conclusion.

    Vote: Thumb up 2 Thumb down 0 You need to be logged in to vote
  18. Adolf Fiinkensein (2,830 comments) says:

    What a pity he took no notice of the empirical evidence which, for some years now, has confirmed the global warming/climate change religion to be nothing but a gigantic scam, founded on flawed computer models and doctored pseudo science.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  19. Harriet (4,614 comments) says:

    DPF – “…..I can think of a number of areas, such as public health, where researchers are impassioned advocates to put it mildly..,..”

    eh?……..like gay health……..surely not ? :cool:

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  20. JeffW (324 comments) says:

    Bill Courtney, the ideological position is one in which the freedom of people/organisations/businesses to set up schools is proscribed, as is the freedom of parents to send their children to a school of their choice. Freedom must be the default setting, the question is why teacher’s unions want to stop freedom applying in their sector.
    Charter schools will either fail, in which case the unions can say they were right, or they will succeed, in which case anyone who is in the profession to help kids should be getting on board and promoting them further.

    Vote: Thumb up 1 Thumb down 1 You need to be logged in to vote
  21. Kea (11,878 comments) says:

    Bugger all policy or law is guided by evidence. Usually is it ideology and appeals to misguided popular sentiment. Both of which can, and are, manipulated by the left wing media. That is why it is rare that left leaning initiatives actually help those they are targeted at. Often their attempts to “help” make things worse.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  22. Rex Widerstrom (5,328 comments) says:

    kowtow suggests:

    No more new tax payer funded jobsworths. Common sense used to be the criteria.

    I’d agree, but I am also a fan of evidence-based policy. So how do we combine the two?

    Simple – make the whole process, from the very first draft of a policy, open to scrutiny and comment. Not just when it’s been decided behind closed doors, got the approval of (and equally ideologically-driven) Minister and people are constrained within the Select Committee process to commenting only on what’s in the draft legislation and MPs effectively forced into a binary (yes/no) response to the entire idea unless they get off their lazy arses and do their own research, which few – especially on the government (any government) benches ever do.

    Have debates just as we have here, open to anyone, on Ministry and quango websites. And just as happens here, anyone who couldn’t back their assertions with at least some links to information would usually tend to be listened to less than those who could. And don’t just restrict discussions to Ministry-initiated proposals – have a “new topic” button that would allow anyone to make an entirely new policy suggestion and have voting buttons to see if the majority of people thought it a common sense suggestion (of course such a loosely controlled online vote couldn’t be used for final decision making, but would be a good guide to the relative practicality vs lunacy ratio of an idea).

    This isn’t new of course – the “wisdom of crowds” has been talked about for ages. But that’s all that’s happened – talk. Yet I regularly see more good ideas here in a day than I hear talked about by politicians in a month.

    An added advantage would be that all the thousands of “policy analysts” employed in government would finally have something useful to do – they could be their departments’ web admins.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  23. Redbaiter (8,009 comments) says:

    COST BENEFIT

    Do a cost benefit analysis of global warming economic measures and it would be out the fucking window in a millisecond.

    All the most gross bullshit.

    Vote: Thumb up 1 Thumb down 1 You need to be logged in to vote
  24. somewhatthoughtful (456 comments) says:

    But DPF, if MSD, MBIE, or NZTA had science advisors they had to listen to National wouldn’t be able to do anything!

    Vote: Thumb up 2 Thumb down 0 You need to be logged in to vote
  25. ChrisM (101 comments) says:

    Griffy

    Did you read this:
    http://www.salon.com/2013/09/05/naomi_klein_big_green_groups_are_crippling_the_environmental_movement_partner/singleton/
    It sums you up to a T. It is also bit of the background to the Gluckman comments. That why he and Roger Peilke quote each other often.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  26. Griff (6,990 comments) says:

    :lol:
    its cris m
    hows the little fire going wingnut?
    The one you maintained was not that big three weeks ago
    its still fucking burning.

    Scientists used to be well represented among the nearly half of Americans who voted Republican. But that’s changed over the years, and one poll found that just 6 percent of scientists call themselves part of the GOP now.

    Vote: Thumb up 0 Thumb down 2 You need to be logged in to vote
  27. ChrisM (101 comments) says:

    Griffy
    I see you are still winning over the undecided with your erudite command of the English language. No doubt, you are one of the Green charm brigade

    Yes – Rim fire is only 80% contained and has burned 237k acres, making it is the biggest fire in California for some years but overall for the whole of US YTD there has burned about half the area of 2012 and 2011. Have a look how big the Siskiyou and Rodeo fires were in 2002. To save you taxing your prejudices, each was over twice the size of Rim. But you never bothered letting inconvenient facts stand in the way of your rants and sprays did you.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  28. Griff (6,990 comments) says:

    The undecided are not my target fuckwits in denail are.
    I am not here to be friends with wingnuts I am here for the fun of rubbishing them.
    You have supplied lots of laughter with your stupidity already come back often idiot .

    Vote: Thumb up 0 Thumb down 2 You need to be logged in to vote
  29. ChrisM (101 comments) says:

    You are so predictable, Griffy. Whenever you are confronted by unassailable facts, you resort to bluster and abuse. And the tighter the corner you box yourself into, the more abuse you throw. I can really see why you are a Green supporter. No one else, even Labour would tolerate such a flake.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  30. Griff (6,990 comments) says:

    Hows the no drought in the usa going ?
    record wet in the east plus record dry on the west Cris M says that means no drought .
    You are the one who is a fuck wit. Hows the argument fire fighting has not changed since the twenty’s going? :lol:
    Brian dead idiot.
    Are you in denial of climate change or just the fact that wildfires are getting more extreme?

    http://www.nasa.gov/content/goddard/nasas-landsat-revisits-old-flames-in-fire-trends/
    http://www.reuters.com/article/2013/08/14/us-usa-fires-idaho-idUSBRE97C10J20130814
    http://news.nationalgeographic.com/news/2013/08/130827-wildfires-yosemite-fire-firefighters-vegetation-hotshots-california-drought/

    Vote: Thumb up 0 Thumb down 2 You need to be logged in to vote
  31. mikenmild (11,246 comments) says:

    I’m waiting for seomone to come out in favour of evidence-free policy.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  32. ChrisM (101 comments) says:

    Oh Griffy, you are starting to calm down. No doubt your mummy made you a nice warm drink of milk. You still are using bluster to cover your incompetence.

    Please don’t use news articles – it just shows you are a cut and paste bullshit artist. What ever happened to peer reviewed literature or official government sites?

    With regards to drought – here is what NOAA says:
    http://www.ncdc.noaa.gov/paleo/drought/drght_500years.html
    and look at the relationship between PDSI and ENSO
    http://spot.colorado.edu/~schoenna/images/Schoennageletal2005RockiesFire.pdf

    With regards to fire policy, here is just the latest I know of:
    http://www.nifc.gov/policies/policies_documents/GIFWFMP.pdf
    or here:
    http://www.bia.gov/cs/groups/xnifc/documents/text/idc013062.pdf
    where they have little gems like this:
    “Wildland fire will be used to protect, maintain, and enhance resources and,
    as nearly as possible, be allowed to function in its natural ecological role.
    Use of fire will be based on land and resource management plans.”

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  33. HC (152 comments) says:

    When reading Gluckman’s report I had to ask myself, how much “science” – and that is credible scientific research and evidence – was accepted and applied by the government and Minister Paula Bennett, when passing the Social Security (Benefit Categories and Work Focus) Amendment Bill into a statute? I wonder how much true scientific evidence is being applied by the Ministry of Social Development?

    As far as this analysis shows, the “experts” used and relied on were perhaps rather reliant on “pseudo science” and selective information gathering, rather than solidly based science:

    http://accforum.org/forums/index.php?/topic/15188-medical-and-work-capability-assessments-based-on-the-bps-model-aimed-at-disentiteling-affected-from-welfare-benefits-and-acc-compo/

    According to Gluckman’s report and a contained table, MSD are somehow “lacking” in applying scientific evidence.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote

Leave a Reply

You must be logged in to post a comment.