Wiggs on League Tables

July 20th, 2012 at 3:00 pm by David Farrar

blogs:

A group of academics signed off on a letter against school league tables. The stated logic may work in an academic research setting but is inappropriate to apply to the real world. We should instead publish the measurements, improve the measurements and their context over time and, most importantly, focus energy and resources on understanding the issues and helping the schools at the bottom of the league.

Exactly. If data is not perfect, them people with a genuine interest in giving parents information and choice will focus on how to improve the data, not call for it to be suppressed.

Some of the more detailed responses:

The argument is that schools have high variability between each other and across years. It’s a combination of measurement error based on inconsistent and low samples and the only measuring numeracy and literacy and not more holistic skills.

However to improve something we first need to measure it, and if we can’t measure it accurately then an approximation will do. In business that means using surveys of customers that have clear sampling bias, reacting more to customers who complain and even believing what we read in the papers. We know all of these sources are incomplete and have bias, but we can account for it somewhat, and are much improved by using the input. The online advertising industry is a lovely example, using a system for measurement that is clearly wrong to measure traffic, but while it is wrong, it is wrong for everyone, and it’s only the starting point for a conversation.

It’s far easier to start a conversation about the quality of a school when confronted with a combination of the socieoeconomic data about the catchment area and the National Standards results over time.

Exactly. Parents are not morons. Few are going to just look at a league table and say we’re going to decide solely on that. Information on how schools are doing with national standards will be just one of many inputs.

I understand the natural academic reluctance to never release data that is potentially wrong, and I see that in business sometimes where companies do not want to release an imperfect product. But while they are polishing the bezels yet again competitors are releasing their inferior but higher selling versions. Similarly we should release the data, and call on the power of academics, hundreds of thousands of parents and even students to provide both sunlight as a disinfectant and the right context.

The answer to bad data is good data, not suppressing all data.

While even a small minority, and this is not a small minority, wants access to our data, New Zealand has a policy and obligation to provide it. Arguing against releasing data is quite remarkable for a group of academics. It should be easier to understand school performance than to read about individual student’s private lives on Facebook.

Most academics support the Official Information Act as a wonderful thing. Educational academics seem to regard it as a bad thing.

Tags: , ,

28 Responses to “Wiggs on League Tables”

  1. YesWeDid (1,048 comments) says:

    ‘Why 107 academics are wrong’ and one management consultant is right?

    Why do people with no teaching qualifications or experience think they know more than those who have devoted their professional life to education?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  2. Reid (16,440 comments) says:

    I understand the natural academic reluctance to never release data that is potentially wrong,

    There is nothing so vigorously defended than a vested interest disguised as an intellectual conviction.

    All teachers of any political stripe at any level, hate falsehood. They’re the first ones to challenge people on the facts and go to the library and check things out. They hate falsehood.

    However, lefty teachers in particular also have another trait: they hate competition. They can’t stand it, because in their twisted heads, competition by definition means dog-eat-dog ruthless to the death struggle and because they’re kind and sensitive creatures why that’s just tewwible and then they start blubbing. That’s what lefty teachers really truly think all competition is. They do not recognise competition has anything beneficial going for it and they think everyone involved in it is always brutalised in the process as an inevitable consequence of participation, regardless of whether or not the participant wins. That’s what they think. I know.

    So lefty teachers have disguised their vested interest with their intellectual conviction. What all the journos need to do which they won’t but they need to, is to ask the lefty teachers why they don’t like competition, instead of asking them why they think the data is inaccurate. But isn’t it mental, that the entire education system at all levels from primary to tertiary is beholden by such towering idiocy as this.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  3. Nookin (3,341 comments) says:

    “Why do people with no teaching qualifications or experience think they know more than those who have devoted their professional life to education?”

    Why is it that only teachers know what parents want, business wants, society wants and everyone needs. Doctors had this mentality 30 years ago. It was a mortal sin to ask a doctor what was wrong with you. Teachers need to realise that more people have an interest in and expectation of education than just teachers. And, for fear that this is seen to be an over-generalisation, I am firmly of the view that the opponents are probably a minority but have the loudest voices and biggest egos.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  4. Nookin (3,341 comments) says:

    I was at a seminar not so long ago. The presenter asked for client classifications —commercial, rural, elderly,first home buyers etc. One attendee (good word that, thanks Penny) put teachers in their own category. Sums it up really.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  5. Mark (1,488 comments) says:

    Good god Farrar do you go trolling for this sort of crap to try to justify the completely illogical. If the data is flawed before you go publishing the stuff and have people relying on it for decisions as important as which schools they should send their children to then you fix the poor data. Fuck me, other professional groups get sued for giving people crap information and suggesting it can be relied upon. It should not be that damn difficult to see. Unlike you i am not comfortable with hanging teachers and schools out to dry based on inconsistent, unmoderated teacher based assessment models. The national standards information is quite different from the NCEA information used for the league tables published about secondary schools yet you have on more than one occasion tried to connect the two. NCEA results are independently moderated and consistently based upon a qualifications authority model.

    The first step for those who want proper objective data to base the assessment of schools performance upon is to have a National standards that is what is is described to be. I am beginning to sound like a broken record on this by the current Tolley system is neither a standard nor could it even closely be described as nationally applied. Christ there is substantial variability between local schools.

    So if the prime minister and his mates are hell bent on league tables then first get a proper objective measure and then have the system independently moderated. Personally I used the ERO reports, went and sat down with the principal and deputy principal of the school I sent my two children to and got a very good sense of where the school was heading. now as they are moving on to secondary school I have been very happy with where they have progressed to.

    Sometimes I wonder about league tables and whether they are genuinely about an egalitarian view of the right wanting to improve the lot of the lower decile schools or is it simply the ability to boast that our kids go to better schools. The cynic in me probably wishes for the former but really believes the latter.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  6. greenjacket (465 comments) says:

    I don’t get it. In almost every profession performance measurement is basic to improvement. Business, sport, police, corrections, public sector – all do it. You identify what matters (and I’d assume that literacy and numeracy were the two main ones for primary school), measure it, and see if you are going forwards (or not).

    Why do teachers think they are above accountability to those who have the ultimate interest in education? The parents whose children are being educated and taxpayers who pay for the teachers are entitled to know that schools are performing or not.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  7. YesWeDid (1,048 comments) says:

    @greenjacket – schools and teachers are performance measured by the Education Review Office (ERO), with reports available online.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  8. Nookin (3,341 comments) says:

    YWD
    Only in very generalised terms. The publicised document is pretty much sanitised.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  9. Psycho Milt (2,412 comments) says:

    The answer to bad data is good data…

    How about, “The answer to bad data is sacking the twats who imposed against advice a model guaranteed to produce bad data, and starting again with a model more likely to produce good data?” The twats in this case are easy enough to identify, they sit around the NZ govt’s cabinet table on a regular basis.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  10. Tom Jackson (2,553 comments) says:

    If data is not perfect, them people with a genuine interest in giving parents information and choice will focus on how to improve the data, not call for it to be suppressed.

    That doesn’t follow. If results are to be presented to a particular group, they ought to be presented when they are sufficiently accurate such that the target group is capable of being properly informed by them. That is not the case with the crude measurements we have now.

    If we want evidence based policy, then there should be no difference between what satisfies academic standards and what’s good for “the real world”. Whether you are for or against league tables, there purpose is to inform the public and they have to have a high enough signal to noise ratio to actually do that before they are of any use. If they don’t, you are confusing the public rather than informing them.

    Having the government publish league tables before the measurements are any good is about as responsible as the government mass distributing warning pamphlets about a link between cell phones and brain cancer when the research indicating any link is at a formative stage. It would be irresponsible and stupid – even if tabloid newspapers were “beating the government to market” by publishing irresponsible scaremongering (that Wiggs thinks this is a good idea marks him out as a corporate shill).

    By all means allow public access to the raw data, but don’t let the government give it any sort of official blessing as a genuine result until such time as the measurements are accurate.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  11. expat (4,050 comments) says:

    Every attempt to start measuring in a standardised manner the performance of kids across the school system is met with reasons why it should never be done.

    The job has to start somewhere FFS.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  12. Tom Jackson (2,553 comments) says:

    The whole league table thing is stupid any way. If you sent a list of every school in your area to every parent in your area and asked them to rank them, you would get pretty much the same results from everyone. People already know with incredible accuracy which schools are “good” schools and which ones are “shitty” schools. Likewise, they tend to know which schools are getting better and which ones are getting worse. For example, everyone in Hamilton knows that Fairfield College is a dysfunctional, low achieving shithole with a student body comprised largely of future criminals and welfare beneficiaries.

    I thought conservatives were supposed to be against government waste. Why are they all for wasting public money on informing the public of what they already know?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  13. expat (4,050 comments) says:

    Um, lets see.

    So poorly achieving schools can be identified formally, steps taken to address issues, improved, remeasured, etc.

    Or shall we all “just know” and do fuck all about it to save the educational professionals ie union whiners some hassle.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  14. expat (4,050 comments) says:

    Oops, post 1700hrs so Tom will be off the clock now.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  15. Mark (1,488 comments) says:

    Expat what the fuck do you think the ERO does but identify poorly performing schools and put in place steps to get them to improve. League tables based on a pick your own testing system and modify the results of you don’t agree is going to achieve what exactly?

    You are shooting the teachers when the halfwits around the cabinet table proceeded with a flawed standards regime despite being asked by the education professionals to test it first.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  16. bc (1,367 comments) says:

    Mark @ 4.14pm
    You’ve nailed it! I did exactly the same as you did. When choosing a primary school for my children, I did the same as you – I did some research by reading ERO reports. Two schools close to where I live sounded good. One was a decile 10 and the other a decile 8. I made appointments to visit the school. The decile 8 school had a wonderful feel to it compared to the decile 10 school. The principal was welcoming, listened to us and asked questions. When she took us around the school and there were children outside, the children said hello to her and she responded with their names. We went into the new entrants classes and saw happy children working away independently and with the teacher.
    We went with the decile 8 school, and are delighted with the school. Told us a lot more than numbers in a table.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  17. mikenmild (11,247 comments) says:

    Well, they wouldn’t want to subject it to a test that they knew it would fail, would they? The standards are a pledge to a few believers; a policy that pretends to so something while not costing much and annoys the pesky teachers and the ‘experts’ in their ivory towers. In short – perfect National Party policy.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  18. expat (4,050 comments) says:

    Mark, they DON’T tell the parents what the hell is going on in most cases, it’s all carefully phrased language mean’t to prevent the parents know what’s going on.

    So counter argument, if it’s no biggie and ERO has it sorted why not humour the dumb ass parents and provide a standardised measure of student achievement across the school network?

    And BC, WTF, choosing a decile 8 school over a decile 10, such a middle class dilemma.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  19. slijmbal (1,236 comments) says:

    Blah Blah Blah

    Measurement has quality and/or evaluation issues and parents are too stupid to take this in to account

    Blah blah blah

    teachers are experts and must be listened to

    Blah Blah blah

    has nothing to do with the political predelictions of teachers

    Blah Blah Blah

    Is it me or is the average teacher quite an isolated individual with little real world experience?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  20. mikenmild (11,247 comments) says:

    ‘Is it me or is the average teacher quite an isolated individual with little real world experience?’

    Is it me, or is there still quite a bit of irrational anti-teacher prejudice in this country?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  21. OneTrack (3,087 comments) says:

    mm – nah, it’s just you

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  22. Reid (16,440 comments) says:

    Is it me or is the average teacher quite an isolated individual with little real world experience?

    Only the ones who haven’t won lotto and thereby turned into ‘rich pricks’ slijmbal. Otherwise no, it isn’t you.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  23. bc (1,367 comments) says:

    It’s you slijmbal.
    I would imagine that teachers come across people from more sectors of the community than most people.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  24. mpledger (425 comments) says:

    I am an (a non-education) academic, statisician and parent of a primary school child. I think the educational academic nailed it but then they pretty much said a lot of the same critisms that I’ve brought up before.

    It’s not well measuered, not well moderated, open to abuse and cheating and is not presented with any guidence on the errors (statistical and otherwise) invovled. As a statistical measure it pretty much fails.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  25. barry (1,317 comments) says:

    In business there is a very good saying – it goes “The customer is always right – even if they are wrong”. Now when customers are wrong, that actually means that the business hasnt yet figured out how to design or make or deliver a product so that the customer isnt wrong. Whe you decide that the customer is wrong – that actually means that the supplier has failed.

    Now parents are customers of the education system. What the teachers and their unions and the academics need to come to understand is to follow the same reasoning. If they think parents are ‘wrong’ to follow the league tables because they have interpreted them incorrectly – that actually means that the preparation of the league table is wrong – and needs to be fixed.

    These arogant people need to take the approach “yes – we will work with the government to make the league tables a decent reliable easy to use product. and to stop being idiots – because the alternate is coming along – and its called things like:
    Remove zones (that will kill those schools thought to be bad – even if the parents are wrong)
    Charter schools (every effort will be made to make these work well – and then everyone will want them)
    amalgamated managment (sucessful school admin will take over badly run schools)

    etc, etc

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  26. Alan Wilkinson (1,878 comments) says:

    mpledger, your fallacy is that humans can only work with statistically valid data. In fact we have evolved over many millenia to deal with imperfect information and make allowances and adjustments for those imperfections.

    So it would be with league tables. There will be debate around the imperfections. Parents will make different choices and discuss the outcomes. There will be mistakes and successes. Academics will write papers. Bureaucrats will expand their empires. Small businesses will create advisory services.

    Life will go on. Generally it will be improved.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  27. mikenmild (11,247 comments) says:

    So it’s better to have bad information than no information?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  28. Alan Wilkinson (1,878 comments) says:

    @mikenmild: Of course it is. That is exactly why pharmaceutical companies explore herbal, folk and indigenous remedies. It is much more cost effective than random experimentation.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote