Is this what the teacher unions fear?
Marginal Revolution blogs an amazing story from the LA Times:
The Times obtained seven years of math and English test scores from the Los Angeles Unified School District and used the information to estimate the effectiveness of L.A. teachers — something the district could do but has not.
The Times used a statistical approach known as value-added analysis, which rates teachers based on their students’ progress on standardized tests from year to year. Each student’s performance is compared with his or her own in past years, which largely controls for outside influences often blamed for academic failure: poverty, prior learning and other factors….
In coming months, The Times will publish a series of articles and a database analyzing individual teachers’ effectiveness in the nation’s second-largest school district — the first time, experts say, such information has been made public anywhere in the country.
One can almost hear the alarm bells going off in NZEI and PPTA offices around the country. You thought school league tables were bad – how about teacher league tables. And worse of all, ones that take into account outside influences, so that they do measure the impact a teacher has over time.
This graphic is not a mockup with fake names and made up data. These are two real teachers, and their real performance.
After a single year with teachers who ranked in the top 10% in effectiveness, students scored an average of 17 percentile points higher in English and 25 points higher in math than students whose teachers ranked in the bottom 10%. Students often backslid significantly in the classrooms of ineffective teachers, and thousands of students in the study had two or more ineffective teachers in a row.
And consider how fiercely teacher unions fight against performance pay.
The conclusion is one that strikes home:
We cannot simultaneously claim, however, that teachers are vitally important for the future of our children and also that their effectiveness should not be measured. As systems like this become more common students will benefit enormously and so will teachers. Moreover, I see this as a turning point. Once parents have this kind of information who will allow their child to be in a class with a teacher in the bottom ranks of effectiveness?
Oh my God. Allowing parents to choose what schools their kids go to. When will this lunacy end.
And the from the LA Times article itself:
Contrary to popular belief, the best teachers were not concentrated in schools in the most affluent neighborhoods, nor were the weakest instructors bunched in poor areas. Rather, these teachers were scattered throughout the district. The quality of instruction typically varied far more within a school than between schools.
This has been stressed back home also.
Although many parents fixate on picking the right school for their child, it matters far more which teacher the child gets. Teachers had three times as much influence on students’ academic development as the school they attend. Yet parents have no access to objective information about individual instructors, and they often have little say in which teacher their child gets.
Parents should not get any choice in which school their kids attend, let alone which teachers they have. This is heresy.
Other studies of the district have found that students’ race, wealth, English proficiency or previous achievement level played little role in whether their teacher was effective.
But it is all about the decile they live in!
No one suggests using value-added analysis as the sole measure of a teacher. Many experts recommend that it count for half or less of a teacher’s overall evaluation.
This is key. How you improve performance on such tests is not the only factor that should be taken into account. It is not a perfect measure. But it is still a pretty damn useful one.
On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. Overall, he ranked among the least effective of the district’s elementary school teachers.
Told of The Times’ findings, Smith expressed mild surprise.
“Obviously what I need to do is to look at what I’m doing and take some steps to make sure something changes,” he said.
Isn’t that great? Rather than get defensive and decry the analysis, he is going to re-evaluate his teaching methods.
And also:
Still, Caruso said the numbers were important and, like several other teachers interviewed, wondered why she hadn’t been shown such data before by anyone in the district.
“For better or worse,” she said, “testing and teacher effectiveness are going to be linked.… If my student test scores show I’m an ineffective teacher, I’d like to know what contributes to it. What do I need to do to bring my average up?”
The real scandal is that it took a newspaper to do, what the education profession could not, or would not, do.
August 23rd, 2010 at 9:11 am
Thanks for publishing that David. It’s very interesting and a good lead-in to the release of performance data in the tertiary sector in the next few weeks. These stats aren’t perfect, as you note, but they add to our understanding.
August 23rd, 2010 at 9:22 am
No. This is all wrong. There is a PPTA ad in this morning’s paper which says the best way of investing in education is to pay teachers more – rather this is a blunt restatement of its content.
August 23rd, 2010 at 9:24 am
Why? What might the other half consist of?
August 23rd, 2010 at 9:25 am
I’m not sure that I would trust the LA Times to do the math correctly or that the math is anything other than double-talk with figures.
August 23rd, 2010 at 9:26 am
If there is an international market for teachers then they could be on to something.
August 23rd, 2010 at 9:28 am
Another view on the source used in the article.
August 23rd, 2010 at 9:29 am
Oh, when the teachers start to pass the students to make themselves look good… what happens then?
[DPF: Doesn't work, as it measures how much a student improves on their past performance]
August 23rd, 2010 at 9:41 am
Thanks cha – now perhaps you’d like to tell us how this analysis is wrong – because as someone who does not delve into this area much, but who does have kids at school – I’d be fascinated to know more about the arguments involved.
I also ask this of you since after reading another view all I found was stuff I already knew about the past of the Bland Corporation [
] – and I had to laugh at this pertinent comment from your link:
August 23rd, 2010 at 9:42 am
Well that might be the obvious answer to my first question at 9:24!
August 23rd, 2010 at 9:43 am
This report is not too dissimiliar to teacher assessments against the National Standards – as summarised in this graph. Again they are real teachers, assessing a real student’s performance against the National Standards. If John Smith ( obviously one of the most ineffective teachers) assessed work against the National Standards, that ineffectiveness would probably not be reflected in that assessment – irrespective of whether the variable of a degree of student motivation played a part on that assessment.
August 23rd, 2010 at 9:53 am
Thanks Cha, can I have the 60 seconds of my life back that you took up in suggesting I read that history of the Rand corporation, which had nothing to disprove the accuracy of the data, apart from RAND =’s Bad in your eyes.
August 23rd, 2010 at 9:53 am
I’ve got two issues with this:
1. The assumption that success as a teacher is measured by the test scores of the students
There is some power to this approach, but it’s far from perfect – there are ways a student can learn, grow and generally be improved that are not reflect in test scores. Focussing only on ‘teaching to the tests’ would crowd-out these benefits.
2. The administration of tests by teachers themselves
A substantial part of NCEA assessment is administered by departments and teachers themselves. If teacher/departments/schools are to be actively and closely judged by their test results, then the ministry must take back all of the administering of assessments that they have passed off onto schools. Otherwise, teachers and departments will ensure the success of students, even if it is not always merited.
August 23rd, 2010 at 9:54 am
Buddin may be absolutely right Tom, about the woeful performance of some teachers, but why the assertion in unison that teacher unions are the problem, teachers pay rates are the solution. Playing ideologue games using kids and their futures as the ball?.
August 23rd, 2010 at 9:55 am
So DPF maybe you could explain to me how this is meant to work, does Miguel Aguilar get a pay increase but John Smith does not?
Does this go on for a number of years until John Smith is so disillusioned with teaching that he gives up?
Do parents get to say I want Miguel as my child’s teacher not John? What happens when Miguel’s class is full, do all the unlucky kids get stuck in John’s class because they are no hopers or their parents aren’t pushy enough?
What about if John Smith is really good at coaching sport or music? Or gets all the rowdy difficult to control children because he is good at controlling them? How is that measured?
Do teachers give up teaching any activity that is not measured and rewarded?
[DPF: Yes Miguel gets a big pay increase. He is worth his weight in gold.John now has feedback on his effectiveness and has pledged to look at how he can improve. If he does improve that is a win-win.If he does not improve, then he needs to be evaluated on his overall performance (not just the SAT improvements).
As it measures how a kid improves over the year, then the fact the kids start off no hopers is not a big issue because if they were say at the 20th percentile and stay there, then they at least have not decreased.]
August 23rd, 2010 at 9:55 am
This is a great post, David.
August 23rd, 2010 at 9:56 am
Buddin may be absolutely right Tom, about the woeful performance of some teachers, but why the assertion in unison that teacher unions are the problem, teachers pay rates are the solution. Playing ideologue games using kids and their futures as the ball?.
Pathetic. Just pathetic.
August 23rd, 2010 at 9:58 am
Netnanny, perhaps?.
August 23rd, 2010 at 9:59 am
All about unions and pay rates Ben, that’s it?.
August 23rd, 2010 at 10:05 am
Hey can we develop quantitative performance measures for cabinet ministers? For Rodney it could be satisfaction with the Super City, for Tolley achievement levels, English unemployment figures, Joyce broadband costs and Ryall DHB budgets. Any other suggestions out there?
August 23rd, 2010 at 10:06 am
That would be what external assessment is for.
August 23rd, 2010 at 10:06 am
So no problems with an “analysis” that was long on Left-wing memes about the dangerous and hidden influences of the US military-industrial complex on US society – in this case the education system – and short on any actual counter-factuals or arguments.
Points deducted for the ad hominem arguments of your link, but points given for the far more subtle, if still standard, technique of debate framing. Other examples:
I care about poor people
….
I support peace
…
I have principles
…
I’m a pragmatist
…
Oh yeah – forgot this one: I’m here to debate the issues
August 23rd, 2010 at 10:08 am
Other studies of the district have found that students’ race, wealth, English proficiency or previous achievement level played little role in whether their teacher was effective.
To which DPF comments:
But it is all about the decile they live in!
I think you’ll find it’s the students’ performance that’s affected by decile, not the teachers’.
Once parents have this kind of information who will allow their child to be in a class with a teacher in the bottom ranks of effectiveness?
To which DPF comments:
Oh my God. Allowing parents to choose what schools their kids go to. When will this lunacy end.
1. We can rely on teacher effectiveness to be normally distributed – ie, however much we improve the effectiveness of teachers, some children will by definition be in a class with a teacher in the bottom ranks of effectiveness.
2. Whose children will those be, do you think? Nice middle-class ones like ours, maybe? Yeah, right. What you’re promoting here is really nothing more than a process for improving the ability of cash to trump equality of opportunity – I’m not surprised by that, because it’s what Tories believe in, but please don’t expect non-Tories to be enthusiastic.
[DPF: Oh I love the hard left. Not only do they want all people to be equally poor, they want all teahcers to be equally bad]
August 23rd, 2010 at 10:10 am
Cha
but why the assertion in unison that teacher unions are the problem, teachers pay rates are the solution.
How does paying them more help?
Why does paying them more help help improve those who aren’t performing?
Especially if the teacher unions hinder the setting up of performance measurements
Surely assessments like this in a mix can help correlate effectiveness both across an age band, school and district/area ?
I’d like to know who is effective and what’s being done to assist those who aren’t.
Similarly I’d like to know where my children are in all the stanines and sub sets of their subjects both in relation to the beginning of the year, to end of it and across the age band in the city and nationally.
August 23rd, 2010 at 10:20 am
Thanks for the sneer Tom, appreciated.
August 23rd, 2010 at 10:25 am
‘DPF: Yes Miguel gets a big pay increase. He is worth his weight in gold.’
That is certainly possible within our current system, Miguel can be made responsible for curriculum development, given management units, made a senior teacher. If you are a male in our primary school system there is a very good chance you will make principal.
Not sure John would share your rose tinted view of rolling up his sleeves and wanting to improve, I know the article says that but when he is not being interviewed by a journalist he might have a different view.
[DPF: Oh God. Don't you just see what you said? The the good teahcer should get loaded with non teaching responsibilities, so he has less time in the classroom. I want good teachers earning $100,000 a year - just because they are great teachers]
August 23rd, 2010 at 10:28 am
http://www.ted.com/talks/ken_robinson_says_schools_kill_creativity.html
worth watching.
Sir Ken Robinson on creativity in schools.
August 23rd, 2010 at 10:31 am
If that were the case we likely could not improve upon the current system in NZ, where zoning has guaranteed the ability of cash to trump equality of opportunity – when it’s the cash used to buy or rent those all-mod-cons, middle-class homes that surround those all-mod-cons, middle-class schools.
August 23rd, 2010 at 10:34 am
we buy our apartheid.
August 23rd, 2010 at 10:40 am
How they measure the “value added” component of a teacher is to calculate the difference between a child’s test score between this year’s and last year’s test.
If the kids in class A scored in the the 85th percentile last year and a brilliant teacher makes them score in the 100th percentile then the teacher adds 15 points.
If the kids in class B scored in the 50th percentile last year and and the teacher raises them to the 66th percentile then the teacher adds 16 points and this teacher is considered the better teacher.
But it doesn’t make sense that the worse teacher made perfect students.
If the statistics don’t make sense then they shouldn’t be used.
[DPF: I think it is more important that a teacher raises someone from say the 40th to the 55th percentile than from the 85th to the 100th]
August 23rd, 2010 at 10:43 am
Fascinating initiative. In meetings all day (AKL, why it always wet up here..?) but will come back to this, as it’s an area I’m very interested in.
August 23rd, 2010 at 10:43 am
…zoning has guaranteed the ability of cash to trump equality of opportunity…
Where I live, zoning has guaranteed that the proletarians of Highbury get to send their kids to the same high school my son goes to. Without zoning, the school would be doing its utmost to replace those proletarian kids with “better-quality” intake from other middle-class suburbs. Get rid of zoning and it ain’t just the parents making choices about who goes where. Right-wingers present a propaganda fantasy that without zoning, the poor would be able to send their kids to the top state schools just like the rich do – it’s propaganda that doesn’t hold up to even cursory scrutiny.
August 23rd, 2010 at 10:48 am
Wrong. The biggest factor impacting student performance is the support and encouragement of their parents, followed by the topic and pedagogical knowledge of their teachers.
Sorry socialist victim-creators, wealth and income has very little to do with it.
August 23rd, 2010 at 10:48 am
Fuck I am glad my kids go to independent schools. We get the teachers we pay for and I am happy to say that they are all pretty damn good.
August 23rd, 2010 at 10:49 am
YesWeDid …… “Miguel can be made responsible for curriculum development, given management units, made a senior teacher. ” All of which decrease his effective teaching time and thus remove him from the comparison. The average then drops a notch and John Smith, with no change or effort moves closer to the average. problem solved. Well done.
It may well be that John Smith would be a better administrator and should be the one removed from the classroom and given alternative duties. Outcome …. the average goes up a notch, and another hopeless teacher achieves the dubious distinction of being “worst-in-show”
August 23rd, 2010 at 10:51 am
I agree Psycho Milt.
Far better the poor and downtrodden are zoned into their pathetic ghettos where communist scum can tell them what to do than have the bastards trying to better the chances of their offspring.
August 23rd, 2010 at 10:54 am
Psycho Milt, in terms of the performance pay proposal, we are looking at the relative improvements from students from year to year. So what relevance does your comments about the students backgrounds have?
PMilt: 1. We can rely on teacher effectiveness to be normally distributed – ie, however much we improve the effectiveness of teachers, some children will by definition be in a class with a teacher in the bottom ranks of effectiveness.
True, but overall the education outcomes will improve from the performance incentives policy. If you disagree, show us how they will be worse…
PMilt 2. Whose children will those be, do you think? Nice middle-class ones like ours, maybe? Yeah, right. What you’re promoting here is really nothing more than a process for improving the ability of cash to trump equality of opportunity – I’m not surprised by that, because it’s what Tories believe in, but please don’t expect non-Tories to be enthusiastic.
No, as you said in 1, teacher effectiveness is normally distributed over all deciles. Actually, teachers in lower deciles have more scope to do well from this proposal, as students in lower decile schools may start with lower standards of achievement, meaning their capacity for improvement is arguably greater.
Secondly, why cant a kid with poor or middle income parents, not have the ability to choose a school which suits them, be it sporting, musical or academically focussed, or one which his father or her grandmother went to. Experience suggests that it was not the wealth of kids that determined which school they got into, but the above factors (after smaller zoning ensured those who lived close to the school could attend).
Sadly, socialists love control and homogeneity and their blunt instruments still dominate the education landscape.
August 23rd, 2010 at 10:59 am
If one wants to see a great example of poor parents and their kids getting screwed over, take a look at this interview and article on the scrapping of a school voucher scheme in Washington D.C
This is not a Left-Right battle in terms of politics – recently Obama and co. have begun to stick it to their teacher union allies in other ways, which is as good an indication as any of the unpopularity of said unions with even Democratic voters in the US. A recent Gates Foundation survey of 40,000 public school teachers (PDF sorry) found them wanting better information about student performance..
But it is a Left-Right battle in terms of ideology, in that the Right want to get the improvements going from the parents & kids up, whereas the Left – even in cases where they want to improve the system, as Obama and his education secretary Anne Duncan apparently do – try to accomplish it from the top-down within the existing system.
August 23rd, 2010 at 11:06 am
Oh boy, this is awesome. Something to really think about, rather than a slogan, for a change.
First thing I thought of was, how will teachers game the system? Will there be social pressure on early childhood teachers to dumb down their students? I cant see how the performance based pay-system would provide an incentive for the early childhood teachers. Of course, it could mean that it becomes rational to pay much more for early-childhood teachers and have a higher standard of entry to that field. This would counter-act the social pressure, and would target resources to what many call the most important learning years.
The second thing I thought was, if the distribution of teacher quality was wider within schools than between schools, doesnt this negate the idea that the quality of (if not only the quantity of) education as a key determining factor in peoples future wealth, income and happiness?
August 23rd, 2010 at 11:12 am
[DPF: Oh God. Don't you just see what you said? The the good teahcer should get loaded with non teaching responsibilities, so he has less time in the classroom. I want good teachers earning $100,000 a year - just because they are great teachers]
Sorry?? So the ‘bad’ teachers should be put in charge of the ‘good’ teachers??
$100K a year would be a 50% increase for most teachers, you really don’t know much about the teaching system do you?
August 23rd, 2010 at 11:14 am
DPF: I think it is more important that a teacher raises someone from say the 40th to the 55th percentile than from the 85th to the 100th
Why?
And, even so, the statistic for “value added” that was used doesn’t descriminate between the two teachers in your example.
August 23rd, 2010 at 11:17 am
…overall the education outcomes will improve from the performance incentives policy. If you disagree, show us how they will be worse…
I don’t really have an issue with finding useful ways to assess teacher effectiveness and act to improve it. My comment referred to the effects of providing that information to parents so there can be a competitive market for whose kids end up with the least effective teachers. It ought to be clear enough whose kids would lose in that competition.
as you said in 1, teacher effectiveness is normally distributed over all deciles.
It’s normally distributed, but I doubt very much that it’s also randomly distributed across deciles. Principals already have plenty of ways of knowing which teachers are more effective, and the higher-decile schools are more likely to attract those teachers.
Secondly, why cant a kid with poor or middle income parents, not have the ability to choose a school which suits them, be it sporting, musical or academically focussed, or one which his father or her grandmother went to.
They can – most schools take a certain proportion of out-of-zone kids. The question is, will the school take yours? ie, don’t delude yourself that you’re in a buyers’ market for schools. What you have to remember is that without zoning, it’s not just the parents deciding which school their kids will go to – the schools are also deciding which kids they’re going to accept. It’s another competitive market, and again, whose kids do you think will lose that competition?
August 23rd, 2010 at 11:29 am
YesWeDid … You don’t know much about what parents want for their kids or what kids expect from their education – do you?
August 23rd, 2010 at 11:34 am
Milt – “We can rely on teacher effectiveness to be normally distributed…”
Not sure we can. It may follow a bell shaped curve, but this doesnt mean it is normally distributed. I would argue that a more “peaked” distribution is prefered to one with fat tails. What we are looking for from a new policy is a positive skew, and one of the important questions about the present situation is whether there is a negative skew.
If we can get a positive skew from the new policy, the students in the lower ranked teachers classes would be closer to the current average than is currently the case.
I tend to agree that good teachers will start to have their skill accurately priced, and that this means they will go where people can afford to pay them the most. But I see this as a feature, not a bug.
1. The population of teachers is not fixed, and any change to the remuneration system will affect it. Paying SOME teachers more will make the whole profession more attractive, increasing the available pool of teachers and allowing consumers to be more discerning.
2. In a system in which the best teachers get picked up by the richest schools (which, dont forget, is what people have been saying is occuring right now), identifying the poor teachers at poor schools is much more valuable. Weed out the bad teachers and the average skill increases. Combined with the point above, weeding out the teachers need not result in larger class sizes or a greater workload.
3. If we can see the value of particular teachers, and if that value can be reflected in their price in the market, redirecting resources towards bad schools would have a much more positive effect than it currently does. As we have seen, throwing money at a problem doesnt work as well as some people think. Extra money, by itself, doesnt help that much if it just results in bad teachers getting paid more. If the extra money can be used to purchase better teachers, the outcome for the students would be much more likley to be positive.
I think the marginal cost of the better teachers being attracted to richer schools (ie the “flaw” you identify*) is outweighed by the marginal benefit of the 3 points above. And the fact that the 3 points above are dependent on the system which allows the “flaw” makes that “flaw” a feature, not a bug.
* And, really, the true marginal cost of the new policy would be how much this problem increases from what is currently the case. Like I said, people have been complaining about private schools getting the best teachers for decades. If it was already a problem, how much could the new system do to make it worse?
August 23rd, 2010 at 11:37 am
How many children must a good teacher improve?
Take a class of say 20 kids, most Maori, as occurs in some schools
Of those a third may only be there a third of the year, which can often happen too.
An excellent teacher may bring the remaining two thirds up to a level that will get them a decent but not high level employment – but keeps them off welfare and out of prison.
Meanwhile another teacher improves kids so they get $ 80k jobs instead of $ 60k.
How do you rate either teacher?
August 23rd, 2010 at 11:40 am
‘YesWeDid … You don’t know much about what parents want for their kids or what kids expect from their education – do you?’
Based on what? I’m a parent (and it may surprise you that) I once went to school.
Just because I think DPF doesn’t know the first thing about our education system doesn’t mean I don’t want my children to have a quality education.
August 23rd, 2010 at 11:43 am
“It’s another competitive market, and again, whose kids do you think will lose that competition?”
So the problem is that good schools will be idenitified and will get more people wanting to attend them than they can take? So what?
With teacher remuneration based on the IMPROVEMENT in a childs performance, the smartest schools will be looking for the kids who are scoring lowly right now but could be improved for the lowest cost. Ahem, isnt that exactly what you want? It would actually be the well-performing student that would discriminated against. (Though I dont think that will even happen.)
The students who would lose out would be those who have low scores that cannot be improved. But they would have lost out under the current system anyway.
August 23rd, 2010 at 11:45 am
http://www.ted.com/talks/jk_rowling_the_fringe_benefits_of_failure.html
Well worth listening to and probably worth talking to your kids about.
August 23rd, 2010 at 11:48 am
Yvette – “How do you rate either teacher?”
Hold your horses. Too many people still dont agree that rating a teacher CAN be worthwhile even if you can do it 100% accurately. One battle at a time, please.
August 23rd, 2010 at 11:52 am
Cha: The piece you suggest gives another view is a classic example of playing the man rather than dealing with the issues raised. It also offers up the “argument from authority” or in-group/out-group dynamics if you wish: “I am (/you are not) part of the in-group, therefore you have no right to say anything about it.” I had hoped it would offer critiques because there are some that can easily be made.
MikeNZ: In theory, paying more should attract more competent people. With those of greater competence entering the profession, there are better choices available and thus there are likely to be better teachers teaching as less competent ones can be turned away. Unfortunately, this is a very limited solution particularly for teaching, as career advancement often seems to lead to less contact with students, eg, junior teacher to teacher to senior teacher to head of whatever often progressing further to some form of management with no teaching contact. Now this is great for managerial types, but as a matter of career progression this seems likely to take many of the better, more experienced teachers out of the classroom.
As to this piece highlighted by Marginal Revolution, I queried it elsewhere yesterday, but what I stated was:
“I’ve just read this – http://www.marginalrevolution.com/marginalrevolution/2010/08/la-times-ranks-teachers.html – at Marginal Revolution and am wondering how NZ’s own National Standards is meant to work. I’ve heard it described variously as ranking teachers and as means of measuring students’ progress, be it linked to the teacher or no.
So, I can understand that standardised testing measures are being introduced, but firstly, how is this seen to be directly relating to measuring a teacher’s performance? From the press I’ve seen from the government, performance in these standardised tests are meant to reflect where the child is at in terms of a standard benchmark for those of their age rather than necessarily the teacher’s ability (although teachers will be measured too in terms of this?).
So is the problem with the way the performance in the tests is reported? As shown in the Marginal Revolution article, it is a percentile ranking that is given, so rather than the child being measured relative to some benchmark, the child is measured relative to other children – but this is not necessarily reflective as to how they are when compared to the benchmarking. (This was also noted in a comment at Marginal Revolution posted by: mattmc at Aug 20, 2010 8:48:54 AM, but it was the initial problem that struck me with such ranking.) Is the reporting going to give a percentile ranking instead of an absolute performance ranking?
Using student results to get teacher results would have to be done with improvement measures, e.g. At the beginning of last year, Student A was at 0.3 relative to benchmark 1. They finished last year at 1.25 relative to benchmark 1. This year they began at 1.3 relative to benchmark 2. At the end of this year they finished at 2.5 relative to benchmark 2. In this case, the student is well above the benchmark and has improved at an increasing rate. Of course, this student may have an ever increasing learning rate. But this should be controlled reasonably well if students are randomly assigned or close to it? Or even if not, there are controls around the interpretation of results based upon what is known about the students.
Of course, there are other problems even if percentile ranking does not occur in that the measurement is being taken from standardised tests – so there’s an incentive to teach to the test rather than teach the actual course. As well as this, there could be problems if the tests were changed significantly each year (hard to figure out trends).
I’m not against National Standards, however. I think there should be some kind of attempt to measure the performance of teachers. If performance isn’t measured, it’s more likely to be suboptimal (incentives problem). I wouldn’t be opposed to both absolute and relative measurements being used with relation to the results of teachers and children. It would be interesting, at the least. We’ve all had shitty teachers, I’m would think (if not, then lucky you), and I know at least in my case that some of them are still teaching. Now maybe they’ve improved, but I find it doubtful, particularly as others have informed me that this is not the case. Such teachers need a wakeup call to either improve or get the fuck out of the profession. National Standards could possibly be that wakeup call in some cases.”
I was informed that in NZ, “Unless it’s changed, it will be reported as below average, average, above average, well above average in general fields, and competencies will be assessed in a simple yes/no way. This avoids most of the failings of overseas attempts to do things like this.” I’m not sure as to the value of such measures and I quibble with the term “average” but only because I don’t know how they intend to use it.
August 23rd, 2010 at 12:00 pm
glubbster (304) Says:
Psycho Milt, in terms of the performance pay proposal, we are looking at the relative improvements from students from year to year. So what relevance does your comments about the students backgrounds have?
PMilt: 1. We can rely on teacher effectiveness to be normally distributed – ie, however much we improve the effectiveness of teachers, some children will by definition be in a class with a teacher in the bottom ranks of effectiveness.
True, but overall the education outcomes will improve from the performance incentives policy. If you disagree, show us how they will be worse…
If the performance incentives policy incentivises the wrong things e.g. incentivising multi-choise tests wont produce the creative thinkers necessary in a knowledge economy.
August 23rd, 2010 at 12:03 pm
That’s part of my issue with the whole National Standards thing being pushed here in NZ. It sounds good but again – it’s a rule imposed top-down on the existing system, much like the Obama approach in the US – and like all such rules in all such systems, it can be gamed. As JiveKitty points out, we’ve got to figure better ways of measuring the teachers.
Anybody who wants to feel good about the possibilities of “good” teachers needs to see Stand and Deliver, where Edward James Olmos gave a great performance as the legendary teacher Jamie Escalante, who surprised the whole US way back in 1982 when 18 of his kids (poor, crap backgrounds) passed the advanced Placement calculus exam.
He died just recently and one of the many pieces I read about him actually followed up on what happened in his school after – and that is not a good news story. It also peers behind the necessarily cut-down story in the movie:
Not just that, Escalante himself began to get hate mail and death threats, The Priniciple who had supported him went back to do a Phd but, upon returning, got placed in charge of supervising abestos removal.
There was also this – which perhaps goes to the heart of the choice problem that Milt highlights:
August 23rd, 2010 at 12:16 pm
P Milt “They can – most schools take a certain proportion of out-of-zone kids. The question is, will the school take yours? ie, don’t delude yourself that you’re in a buyers’ market for schools. What you have to remember is that without zoning, it’s not just the parents deciding which school their kids will go to – the schools are also deciding which kids they’re going to accept. It’s another competitive market, and again, whose kids do you think will lose that competition?”
Not true, its zoned kids, then a ballot box. So my point about lack of choice for kids to go to their school of choice stands.
As I said in my earlier post, it will not necessarily be the poorer kids, if they are good academically or at sports, as many are.
Also, remember, lower decile school have greater government resources and still have the ability to attract good teachers. Paying teachers for their teaching may encourage a number to join these schools.
As for the league tables, so long as they are published accurately, I cannot see the problem. I do not believe in hiding information when everyone knows which schools do not perform well in any event! The MOE has no right to hide these figures. Overtime, it will reflect improvements in schools and those who are going backwards.
mpledger, disagree, at lower levels, multichoice tests are great as they lessen the burden of assessment which means more time for teaching. They suit Math subjects where the answer is right or wrong and not until university does it branch out into pure maths, economics or engineering when maths is used to justify research and policy positions etc.
Do you think standard based assessments are any better for “producing creative thinkers” in subjects like english and history! Where is the holistic approach needed, when standards are specifically compartmentalised?
August 23rd, 2010 at 12:17 pm
Oh god – in the middle of an education debate I’ve spelled Principal wrong – twice.
August 23rd, 2010 at 12:20 pm
Kimble – DPF seems to want to allow rating teachers.
Your “Hold your horses. Too many people still dont agree that rating a teacher CAN be worthwhile even if you can do it 100% accurately. One battle at a time, please.” shouldn’t stop my asking “What information or circumstances do you need to be able rate a teacher.”
I completely allow myself to point out this may be a little difficult right from the outset.
August 23rd, 2010 at 12:27 pm
Sorry to digress, but will anyone create today’s General Debate thread?
August 23rd, 2010 at 12:28 pm
Remove State from education…problems largely solved.
August 23rd, 2010 at 12:30 pm
Manolo
To be honest I hope not – they so often degenerate into crap. At least with something like this people stay (mostly) on topic.
August 23rd, 2010 at 12:33 pm
‘Do you think standard based assessments are any better for “producing creative thinkers” in subjects like english and history! Where is the holistic approach needed, when standards are specifically compartmentalised?’
I think the correct education-speak for ‘holistic approach’ is a ‘rich education’, which is not to be confused with being rich and getting an education.
Children need a rich education that is not confined to just passing exams in maths and reading. Yes they need to learn maths and reading to as high a level as possible but child also need to learn to extend themselves creatively, physically and intellectually.
August 23rd, 2010 at 12:38 pm
‘Remove State from education…problems largely solved.’
Let’s leave our education to private schools and religious schools. Let’s make the quality of the education our children get totally dependent on what their parents can afford.
If people are poor and can’t afford a good education for their children, tough, they can stay poor. That will get us ahead, that will allow us to catch up with Australia.
Shit James, you got any other great ideas?
August 23rd, 2010 at 12:41 pm
“There was also this – which perhaps goes to the heart of the choice problem that Milt highlights:”
Escalante gave up. Because he couldnt help everybody he decided to help no one. Thank God everyone doesnt feel this way there would be no charity anywhere. This isnt really a problem of choice.
August 23rd, 2010 at 12:42 pm
glubbster (305) Says:
mpledger, disagree, at lower levels, multichoice tests are great as they lessen the burden of assessment which means more time for teaching. They suit Math subjects where the answer is right or wrong and not until university does it branch out into pure maths, economics or engineering when maths is used to justify research and policy positions etc.
Maths in education is not about getting to the right answer especially when it can be done by blind luck in a multi-choice test. It’s about knowing how to get to the right answer.
If you are doing testing to improve learning then you have to know where the child failed – did they misread or fail to understand the question, transpose the answer in their mind or not have a clue. None of that you can get from a multi-choise test.
And from what I understand, maths multichoice exams tend to pick winners out of those with high comprehension scores not necessarily maths skills.
August 23rd, 2010 at 1:12 pm
The left’s solution to many of the problems facing modern western nation states, (like Education and Health), seems to be to simply throw money at the problem as if they automatically mean better outcomes.
At least on the right of the spectrum alternatives are being looked into. What I am especially interested in at the moment is the movement in the UK to reintroduce Grammar schools. There seems to be a lot of people who now argue that abolishing them was a bad idea.
Instead of a one size fits all policy that the left is keen to push, (potentially because of their equality of everything argument as well a social engineering), maybe it is better to direct the more academically inclined students to schools which will challenge them and have the rest of the State sector focusing on getting children ready to be productive members of society.
August 23rd, 2010 at 1:13 pm
“Maths in education is not about getting to the right answer especially when it can be done by blind luck in a multi-choice test.”
Asking multiple questions eliminates the luck factor. What are the chances of someone randomly choosing 20 correct answers in a 40 question exam? (Show your working.)
“None of that you can get from a multi-choise test.”
Multiple choice can BETTER identify specific problems in learning the process. You can set up the questions so that each possible answer would be a correct answer if a certain mistake is made.
Maths multiple choice exams also help to teach. If a student gets an answer that is not listed they instantly KNOW it is wrong, and they can adjust their calculation.
August 23rd, 2010 at 1:17 pm
We all want the best for our children and we want the best environment for them to learn in.
We also all want the most effective teachers to be rewarded for being so.
That only happens if we are able to look at all the relevant data to discriminate and make decisions accordingly.
If the money followed the child that would happen in that parents would have the power to move their children to the better schools. ( that should include independent schools too, so the state funding could offset the fees).
The Principals and Teachers would be on notice to be transparent and open as their hold on the sector would be greatly reduced.
That must be good for the kids, after all that’s what we want isn’t it, what’s best for the kids?
August 23rd, 2010 at 1:17 pm
Oh and determining reading comprehension in a maths exam is a little silly, dont you think? Arent there reading exams for that?
Having a child fail because they couldnt understand the questions could be identified if you have a possible answer that looks correct to them if they DID misunderstand the quesiton.
August 23rd, 2010 at 1:40 pm
This brings me back to the same fundamental problem that plagues Public Anything. The idea that if competition is introduced into what is a “right”, then people will get screwed, or at least more screwed than they are by the current system. After all, people getting screwed was what gave rise to the public good in the first place, so no need for further debate. Just keep pouring money into the public system, fiddling with the controls, making the rules and regulations ever more precise, and you’ll improve.
What the left has to recognise is that public education simply hides the competition by transforming it into something else. So what we have now is competition between parents to get into the zones of “quality” schools (they’re not always right about the “quality”actually) by spending money to buy or rent the required house.
Supposedly this less than ideal situation (we must try to make every school equal) is better than one where the same parents spend their money directly on the public or private school they (or their kids) want. Note that since the competition has been converted from one where the parents spend money directly on education to one where it is spent on a house, poor people will (and do) lose out, since the latter requires far more wealth than the former.
Apart from transforming the inter-school and inter-parent competition there is another marketplace effect that is hidden – what happens to the kids that a public school has to take. They get streamed/dumped into classes where they don’t get the best teachers and where the expectations are lower.
Whose kids do you think lose both competitions?
Aside from the disparities in the wealth of the people making the choice, what of the argument that the schools get to choose as well and their choice will be the kids who make them look good and garner them more such children and parents in a virtuous circle (with the poor schools in a non-virtuous death spiral). Not to mention that fact that – as with Escalante – success will lead to a rapid mismatch between the teaching resources that seem to succeed and kids and parents who want that.
Those are both valid points – under the current or even the proposed systems of state testing and funding – and that leads me to these two comments:
That’s pretty harsh. Escalante got buried by a system that could not cope with the demand that his successful methods generated. As the article points out, he did not get a huge range of choices, he got two – deal with bigger classes than he thought could work with his methods or use teachers who weren’t up to his standards. What was he expected to do, work himself to death?
It is a matter of choice – but the solution lies in Milt’s unspoken question
Too true. The question is why is it not a buyers market? The question answers itself – because the demand for quality education outweighs the supply. The reason is that the way schools are set up (which is basically still the 19th century industrial model with a lot of cosmetic surgery) their resources cannot keep up with the demand generated by the likes of Escalante – and that means they have to ration, which ultimately degenerates to selecting kids. Given the way our education system currently works that’s probably always going to be the case if we simply switched on the “school choice” option and changed nothing else.
So – how can the system change so that resources meet the demand – or at least don’t fall so far short that we have to ration and that can keep increasing with the demand? I’d suggest changing the methods by which the students are connected to the teachers and change the incentive measures. Kimble already highlighted one possibility in the latter area:
As far as the methods are concerned – communications technology is upending virtually every other industry in the world – maybe it will upend how we educate our kids as well?
August 23rd, 2010 at 1:42 pm
I’d want ‘good’ managers put in charge of teachers.
Good teachers paid more.
Mediocre teachers made aware of their performance and helped to improve.
August 23rd, 2010 at 1:43 pm
MikeNZ
Don’t be silly, the two most important groups are ignored when we just look at what is best for the kids. The unions and the dept of education are more important than the kids. If we measure teachers we undermine the unions and if we remove geographic based zoning we make the administration more complex for the dept. of Ed.
August 23rd, 2010 at 1:45 pm
The unions are classic… Good teachers can earn more by moving from the classroom… Yeah, tell me again how promoting good teachers away from the classroom helps the standard of teacheing…. It lowers it…. But don’t tell the unions that because in their fantasy world all teachers are the same.
August 23rd, 2010 at 1:46 pm
Kimble – nb: I’m arguing against teacher league tables for parents, not against establishing measures of effectiveness for teachers.
So the problem is that good schools will be idenitified and will get more people wanting to attend them than they can take? So what?
According to the article, this is something that would be played out at the level of individual teachers, not schools. And as I pointed out initially, it’s essentially a way of improving the existing ability of cash to trump equality of opportunity.
Like I said, people have been complaining about private schools getting the best teachers for decades. If it was already a problem, how much could the new system do to make it worse?
Parents inevitably want to get the best education for their children and money is a means of achieving that. However, society as a whole wants equality of opportunity when it comes to education, so seeks to reduce the ability to use cash to influence the outcome – decile-based funding and zoning are examples of that. Naturally, and obviously, such measures are of limited success in tipping the balance towards equality of opportunity. You seem to be suggesting that given the limited success of such measures, it won’t make much difference if we just give up and say “Nah, fuck it, let the money decide.” I think it would actually make a very big difference, and not a good one.
So my point about lack of choice for kids to go to their school of choice stands.
You’re still under the impression that you’d be the one doing the choosing.
I do not believe in hiding information when everyone knows which schools do not perform well in any event!
Read the article again. It’s not about schools, it’s about teachers. In fact, the article specifically points out that the school is way less important than the teacher.
August 23rd, 2010 at 1:48 pm
An interesting podcast about teacher performance from Freakonomics Radio – with input from the New York City Schools Chancellor and the US Secretary of Education.
It’s a comment on the 1 classroom, 28 kids way of teaching and an alternative to it.
August 23rd, 2010 at 1:54 pm
@tom – My biz partner and I are working up something new in this space. If you have anything more than a passing interest in this and are keen to discuss then drop me an email at krazykiwi.kiwiblog@gmail.com anytime. cheers.
August 23rd, 2010 at 1:56 pm
So what we have now is competition between parents to get into the zones of “quality” schools (they’re not always right about the “quality”actually) by spending money to buy or rent the required house.
Like I told Kimble – the fact that current attempts to tip the balance away from money and towards equality of opportunity are inevitably of limited success doesn’t mean we should just give up and go with letting the money talk.
August 23rd, 2010 at 2:01 pm
“is this what the teacher unions fear?”
David Farrar subliminal message number 666 – if you join a union you are lazy and stupid…. Good for sowing reactionary attitudes and duping workers in to thinking that they have more interests in common with the boss than the people they spend their days with…
August 23rd, 2010 at 2:15 pm
Professor Warwick Elley, who is a highly regarded expert in the field of educational assessment; identifies some legitimate issues with the proposed national standards policy. http://www.nzpf.ac.nz/sites/default/files/Warwick_Elley_20_Flaws_NS_Policy.pdf
August 23rd, 2010 at 2:44 pm
Like I told Kimble – the fact that current attempts to tip the balance away from money and towards equality of opportunity are inevitably of limited success doesn’t mean we should just give up and go with letting the money talk.
Yes…that would be too close to allowing the people to talk….and we can’t have that eh Comrade?
August 23rd, 2010 at 3:06 pm
James
Only the self serving administrators wanting an easy life and the unions wanting maximum membership numbers should be allowed to set policy.
August 23rd, 2010 at 3:07 pm
Some of us can tell the difference between people and money, James – it’s quite helpful to be able to do this, so I advise practicing it until you get the hang of it.
August 23rd, 2010 at 3:21 pm
KrazyKiwi
Thanks. Unfortunately (and perhaps incorrectly) this whole thing is only a passing interest to me. My field is solid IT and I can’t help thinking that you need people with an passionate interest in educating kids rather than a techie, even a business-oriented one.
However, you might appreciate these two articles:
- Higher Ed’s Tech Dilemmas
- academic luddites
August 23rd, 2010 at 3:30 pm
Eee oop laad. Jus orf down t’ yon dark satanic primary school t’ see Lady Principal ‘boot recent cut in her vege scrap allotment for Year 1 teachers.
Aboot time we made it clear to ‘er that times are a’changin. 20th century ‘ill be ‘ere soon.
August 23rd, 2010 at 3:57 pm
MP said
# Kimble (1,931) Says:
MP>>> “Maths in education is not about getting to the right answer especially when it can be done by blind luck in a multi-choice test.”
KIMBLE>> Asking multiple questions eliminates the luck factor. What are the chances of someone randomly choosing 20 correct answers in a 40 question exam? (Show your working.)
MP> It’s a binomial distributional setting with parameters n=40 and p=1/4 which means P(X=20) = 40!/(20!20!) (1/4)^20 (3/4)^20. Is closed form enough? (BTW that is a trick question).
MP>>>“None of that you can get from a multi-choise test.”
KIMBLE>> Multiple choice can BETTER identify specific problems in learning the process. You can set up the questions so that each possible answer would be a correct answer if a certain mistake is made.
MP> Really! I think there is going to be a lot more ways of doing it wrong then the three options available. That still doesn’t get around comprehension issues.
KIMBLE>> Maths multiple choice exams also help to teach. If a student gets an answer that is not listed they instantly KNOW it is wrong, and they can adjust their calculation.
MP> In the American system, IIRC it’s about 40 secs a question so mucking about if you work it out wrong is very costly in terms of answering the remaining questions.
MP> I’m quote happy to say that Multiple Choice questions are efficient testing mechanisms (and the Australian approach to national standards testing is way better than NZs iff league tables are an outcome) but noone should be fooled into thinking they are testing more than a very narrow range of ability or that they are a learning tool – unless there is reasonable feedback at each answer point.
August 23rd, 2010 at 4:00 pm
Honest John, I don’t see how you can make the leap from “is this what the teacher unions fear?” to ” … if you join a union you are lazy and stupid”
But the latter is an idea worth repeating.
August 23rd, 2010 at 4:04 pm
Sorry Mr Farar as the most popular blog, you are a success, BUT YOU ARE NOT THE PICTURE OF HEALTH, stocky bulky boose to much, eat to much, we post and read , will i end up like you ???? um much older and skinnier, lets do a review of blog sites David and Cameron, shit i would be dead in my 50s. looking at you two.DEMERITES FOR THE TRUTH,like education,there are stats and the truth, what you use is where the funding comes from
August 23rd, 2010 at 4:07 pm
Roger rabbit…
A classic example of why we need National Standards.
August 23rd, 2010 at 4:09 pm
Yeah – because letting money talk has proved such a crap way of getting food, light, heat, communication, books and ever-greater amounts of information into people’s lives – education is fundamentally different of course.
Money is just a name for measuring the cost of resources, of which people are the most important since they tend to create a lot of the other resources, so in a sense……….
But I see the problem. You think that if we hide the money away from the process, siphoning it away from people via taxes and then having it magically reappear in the form of “free”, public education, we will have separated the money and people. We’ll be able to just talk about people, their needs and how they can be met inside the “free” system – and keep that nasty money stuff away off in the background.
Hey – that’s almost like this earlier comment in the thread:
I care about
poorpeopleAugust 23rd, 2010 at 4:13 pm
Some thoughts from one who has been there:
Teachers hate being assessed because they know how shonky a lot of their own marking is. They also know they aren’t responsible for the good, bad and ugly that arrive in their classrooms.
The best principals have always been bloody good classroom teachers.
The best teachers can teach anyone – good, bad and ugly.
Pay good teachers much much more (yes at least $100,000) and then ask them how many kids they would guarantee results for. We would be amazed at what the best could do with great incentives.
Stop allowing the utter disgrace of EIGHT years primary schooling producing kids unprepared for the literacy and numeracy demands of high school. No accountability AT ALL.
Stop letting teacher unions pretend their pay demands are for the good of the kids. BS.
August 23rd, 2010 at 4:16 pm
Thanks Milt.
“According to the article, this is something that would be played out at the level of individual teachers, not schools.”
This makes my point even stronger, as teachers would follow opportunities. They will want to work at schools that have access to children that are under-performing but can be turned around. This is exactly what you want.
“its essentially a way of improving the existing ability of cash to trump equality of opportunity.”
Perhaps, but you havent shown how this results in a reduction in opportunity for others. It really does seem like you are only thinking about the immediate impact of such a change. As if the changes would take place in a static state.
Do you accept that an increase in the potential pay for teachers would attract more skilled people to the profession? I mean, this has been the argument for teacher pay increases in the past; increase the pay to attract better people. Has it worked? The thing is, the base pay doesnt have to be a high guaranteed amount to get the benefits of attracting more people. I really think, once it becomes recognised that the best teachers can get paid a lot, people will enter the profession.
“However, society as a whole wants equality of opportunity when it comes to education,”
Society also wants free, costless healthcare of the highest standard for all who want it. Not going to happen though. With limited resources, it can never be done. Equality is not a worthy goal, in and of itself. Equaly shitty education is not better than unequal education of the same average. Too many people have made this point for it to need to be restated.
Side point: I would go as far as saying that equal education restricts social mobility. If everyone had the same education, there would be less chance for a poor person to excel relative to wealthy people. They would no longer be able to educate themselves up the social ladder.
Equallity of opportunity is good, and it exists now, and it would continue to exist in the future. Everyone has the equality of opportunity to buy a Porsche, some people just cant afford it. Everyone has an equal opportunity to purchase the car, it is the inequality of means that restricts it from some.
“You seem to be suggesting that given the limited success of such measures, it won’t make much difference if we just give up and say “Nah, fuck it, let the money decide.” ”
I am suggesting that, though not in those words. Surely the cost of any policy change in this area is best described as the additional cost relative to the status quo, rather than in relation to some utopian ideal that has never existed anywhere ever?
August 23rd, 2010 at 4:33 pm
MP, how is it a trick question?
If failing the exam, as determined by not getting half the answers right (a low hurdle rate IMO, increase that # to 21 instead of 20 and see how much the probability drops by), indicates that a student has not mastered the course, and if the chances of selecting enough correct answers by chance is ridiculously low, then surely the probability of passing a student who guessed his way to passing the exam is also quite low. Right?
“Really! I think there is going to be a lot more ways of doing it wrong then the three options available.”
True, but it can identify SPECIFIC problems, which is what I said. Addressing specific, or common, mistakes is the low hanging fruit. Pick them up early, correct them, retest, job done. When teaching a class, knowing specificly which techniques to reteach to gain the greatest advance in understanding is very useful
“That still doesn’t get around comprehension issues.”
And like I also said, comprehension is tested in English tests, not maths.
“In the American system, IIRC it’s about 40 secs a question so mucking about if you work it out wrong is very costly in terms of answering the remaining questions.”
Maybe, but at least they have the opportunity to do it. It is instant feedback, and feedback is good in education.
“but noone should be fooled into thinking they are testing more than a very narrow range of ability or that they are a learning tool – unless there is reasonable feedback at each answer point.”
Testing a very narrow range of ability is good, in elementary mathematics. The assumption is that these are on-going tests through out the year, not the final exams, and that reasonable feedback is provided.
August 23rd, 2010 at 4:43 pm
Yeah – because letting money talk has proved such a crap way of getting food, light, heat, communication, books and ever-greater amounts of information into people’s lives – education is fundamentally different of course.
You write this as though the view that the education of children is fundamentally different from providing groceries were in some way odd, rather than the norm in every single developed country.
You think that if we hide the money away from the process, siphoning it away from people via taxes and then having it magically reappear in the form of “free”, public education, we will have separated the money and people.
No, I just think that there should be something more to it than “How much money you got?” James (and you apparently) thinks this is an excellent determiner of who gets what kind of education; every developed country in the world thinks otherwise, for glaringly obvious reasons.
Do you accept that an increase in the potential pay for teachers would attract more skilled people to the profession? I mean, this has been the argument for teacher pay increases in the past; increase the pay to attract better people. Has it worked?
Like I said, I don’t have a problem with attempting to measure the effectiveness of teachers and paying them accordingly. I have a problem with setting parents up in competition with each other for teachers via league tables, which will only exacerbate the problems with equality of opportunity in education that right-wingers love to point out about the current system. It’s not clear to me why, if the problems in the current system are so awful, right-wingers think making them worse is no big deal.
Everyone has the equality of opportunity to buy a Porsche, some people just cant afford it.
This is James’ and Tom Hunter’s point. As a basis for education policy, it sucks a big one and no govt in the West is interested.
August 23rd, 2010 at 5:38 pm
“As a basis for education policy, it sucks a big one and no govt in the West is interested.”
Ah, then it is not the equality of opportunity you want, it is redistribution by way of the education system. That is probably where the confusion lies.
August 23rd, 2010 at 5:56 pm
Some of us can tell the difference between people and money, James – it’s quite helpful to be able to do this, so I advise practicing it until you get the hang of it.
Shame you are unaware that money is how People talk to one another in the market….its how they incentivise others to provide them what they want.I suppose you think its all done by love huh Milty?
August 23rd, 2010 at 6:42 pm
Hey JIVE KITTY!
you may want to elaborate how National Standards measures student performance. I maintain it doesn’t, it only measures it against certain undefined standards. If a certain performance is measured by different teachers accurately and consistently, who then provide different results as measured against the standards, it is hardly an accurate measure of student performance against these standards . Policy fail!
August 23rd, 2010 at 6:55 pm
As I’ve said before we found one of the teachers at the local intermediate to be useless as our son (who is doing five scholarship subjects this year) was having problems with her and she never bothered to contact us. We heard from another parent that parents nicknamed her Dozy Dunlop. Anyway, my wife has rung the school with each subsequent child and asked that they not be put in her class and the office person has been totally unsurprised and didn’t question my wife at all!
August 23rd, 2010 at 7:09 pm
Ah, then it is not the equality of opportunity you want, it is redistribution by way of the education system. That is probably where the confusion lies.
I suspect the confusion actually lies in your assumption that equality of opportunity equates to wealth-based rationing.
…money is how People talk to one another in the market…
I know this will be very confusing for you James, but the Glibertarian utopia only exists in your fantasies. Only a very small part of the NZ education system is a market. Still, why let irrelevance stand in the way of your comments? You never have before now.
August 23rd, 2010 at 8:17 pm
The top quotes for me:
That must drive the uber-socialists nuts. How can they play victim-makers using race and wealth levers in the face of this kind of evidence?
..and..
One of the teachers, who was discovered to be under-performing is quoted in the LA Times:
“For better or worse,” she said, “testing and teacher effectiveness are going to be linked.… If my student test scores show I’m an ineffective teacher, I’d like to know what contributes to it. What do I need to do to bring my average up?”
The teacher wants to improve her performance. Nope, no victim there either. Great stuff!
August 23rd, 2010 at 8:22 pm
krazykiwi
Given the point I made earlier about where general education may be going and what you’re doing, you may also be very interested in this article: A Self Appointed Teacher
August 23rd, 2010 at 8:32 pm
@tom – hey thanks for these links. Salman Khan is an interested story. Have seen a few of his vids. I’ve been doing lengths in education-next stuff today … devoured a few hours worth of podcasts and hit 20+ relevant stories (eg School of One). This space is definitely hotting up, and there are to my mind, still plenty of opportunities to wire technologies together uniquely.
August 23rd, 2010 at 8:40 pm
The best technique I ever heard was Edward De Bono explaining how when he was a lecturer he would give his students the actual final exam in his first lecture.
He used to say to his colleagues when questioned, I’d put my exam and/or students up against yours, any day. And he’s right, that’s what education is. The fact some people fail the final test and the calamity befalling thereof, appears all the current education professionals consider. How mind-bogglingly out of touch with actual humanity can one be?
Oh the humanity.
August 23rd, 2010 at 9:59 pm
Ah yes. I thought this might come into the picture, which is why I put the bit in about books, information and communication. I wasn’t here in the 1990’s but I understand some plonker of a right-wing politician made the comment back then that there was no real difference between baked beans and education – and the left jumped all over this.
Education is not the same as baked beans, but then neither is a million-part airliner like a bag of groceries – yet the latter two are subjected to the laws of supply, demand, and resource pricing and allocation in marketplaces just like any other non-natural right.
If other complex goods and services can be allocated to people via a marketplace then why not education? It involves communicating information and making sure that people have retained it. So what is it about education that is fundamentally different ? The methods by which it’s best delivered to children? Learning how to think about the information and sift through it? Gaining skills through practicing them? The physical presence of a coordinator and guide called a teacher? What techniques does it require that are known to no marketplace?
It seems to me – and this is circling back to the point I made earlier – that simply stamping one’s foot to declare that something is a “right” becomes an excuse for thinking that marketplace stuff does not still apply. But it does – it’s always there, like a river under the ice. Refusing to think that it’s still there and that we can now command it by imposing a “free” system over the top, is how we wind up in the crap world of rationing and regulations and schools checking that people are truly “in-zone” – and yet still with the results of good teachers wondering why they’re paid the same as poor teachers, failures at certain schools that already damage kids and require radical shakeups the look pretty much like the worst-case scenarios imagined about a private-sector education system. Then there’s the endless demands for more money at rates that can exceed the population growth or CPI.
Arguing that we should regard education as a marketplace and that market features should be out in the open is not the same as the reductio ad absurdum that you imply – a devil-take-the-hindmost world where the poor cannot get their kids a good education, leading to them being poor and so on in circular trap. Ironically in many developed countries (including here in NZ) that is exactly what you do get anyway. If it was a marketplace we’d call it market failure.
Which might be true if the only product and standard was a Porsche, and that in this zero-sum world, only those with the money will get it – while others get nothing or a Reliant Robin. Indeed, the further up the education scale you go the more the “Porsche” scenario applies. There are many poor kids who get scholarships to Oxbridge and the Ivy League – but there are many more who don’t.
You don’t have to trust the word of a nasty right-winger that I care about poor people – you should be able to accept that I think that our societies have become so complex that we have to educate everybody to the limits of their capabilities. I don’t think our systems of industrialised public education are delivering that: they may have 100 years ago (or even 70 years ago) when the “three R’s” were the minimum grade to achieve, but they seem to be lagging (and I’m not the only one who thinks that) – and as society changes with ever greater speed I think they’ll lag more, especially for the poor of places like Otara or West-side Chicago.
We’re talking about Year 0-12 education, and the product is already fairly standardised. Just because I want a marketplace of education in these years does not mean that I’m simply going to shrug my shoulders and accept that some kids will get little or nothing because they’re poor. A marketplace does not have to mean chaos. If society has imposed a standard level of education to be attained then it will have to impose a rule that says that everyone has to get up to that standard. We have all that now, but we refuse to really let a marketplace deliver it for us as a society. If we did I think it would do what marketplaces often do – produce something better than just a standard product.
For all the facetiousness of the baked beans comparison there is an underlying point there: if getting food into the mouths of hungry babies can be made to work by allowing farmers (and big agribusinesses) to compete and make a profit then why the hell can’t even the poorest be well educated by allowing schools to make a profit off doing so?
Why can’t they all (or even most) be like this – and why does the Left really think we’re going to get poor kids into those sorts of schools by tweaking the current system with command and control methods instead of marketplace methods?
August 23rd, 2010 at 10:23 pm
That Freakonomics podcast was one of the one I listened to today. Recommended listening folks, skip the intro about radio stuff. The achievement leaps experienced by the NYC kids sounded pretty amazing. Could we do that here? Of course.. if we invite the NZEI exec (and a few uber-lefties) to a retreat on the Chatham’s, and then abandon them there.
August 23rd, 2010 at 10:25 pm
Only the Chathams? I suggest further afield, say urbane and civilised Somalia.
August 23rd, 2010 at 10:29 pm
Or Isla San Lorenzo
(almost certainly too civilised…)
August 23rd, 2010 at 10:36 pm
It would be handy if those commenting could disclaim “teacher”, “unionist” or worse still “teacher unionist”. Cheers. Would help not having to read posts from the latter two.
August 23rd, 2010 at 11:01 pm
@Dave: I think that was my whole question, really. I had problems with the measurement in the article linked to by Marginal Revolution and wondered how the evaluation of teacher performance in terms of student performance was to be done here. The only response I got was from a friend and I detailed it in my post above. I also noted my worry as to the value of what was detailed and my quibble with the use of “average” which may take it back to the problem I had with the use of percentiles as it is a comparative measure, rather than a measure of absolute student performance.
As I think I said, I’d like to see both absolute and comparative measures used to consider a number of variables. Student performance relative to the benchmark. Student performance relative to previous years (rate of improvement) in terms of distance from the benchmark. Student performance relative to the average student in terms of both benchmark and rate of improvement. If environment, outside teaching is a problem, i.e. significant differences between the performance of students in different deciles, I’d like to see some statistical analysis for that as well. So having the measures in raw terms as above, but also giving some adjusted terms to deal with decile/demography/whatever. If decile/demography/whatever is shown to have significant differences, there’s a bonus in teachers not being penalised unfairly for that but also in that these problems can be highlighted and possibly better dealt to than they otherwise would have been.
If you’re wondering about benchmark implementation, I would have examination at the beginning of each year, in the middle and at the end. Externally marked. I think there’d need to be something other than examinations as well, but I don’t quite know what.
I don’t think that kind of thing is actually going to happen though.
August 23rd, 2010 at 11:04 pm
If Parliament used the PPTA system, who would be the best-paid MP…
George Hawkins?
Applied to sport, the best-paid All Black would be the thirtysomething guy who’s just about to hang up his boots, and a 20-year-old Jonah Lomu would get chickenfeed.
Just nuts, isn’t it — all the moreso when these selfish teachers have no difficulty grading their students on merit.
If classes ran on the teachers’ pay system, all the kids born in January would get Excellences, and the bright young geniuses who’ve skipped a class would score Achieveds.
August 23rd, 2010 at 11:14 pm
and
It’s interesting to see the comments from certain posters fixated on ideology rather than results. And even those who are seemingly focused on results ignore the bigger picture. I have yet to see, although I may have missed the odd comment, a poster invoking the opposite view i.e. “uber-facists” or “uber-righties” to that expressed above. As for results:
OECD Education at a Glance 2009
Socio-economic status: In virtually every country for which there are comparable data, students in the top performing category come from families with comparatively advantaged socio-economic back-grounds…There are at least two reasons for this linkage: First, families with more educated parents, higher income and better material, educational and cultural resources are better placed
to provide children with superior educational opportunities both at home and outside the home. Second, such families often have much more choice over where they can enrol their children. However, a disadvantaged background is not an insurmountable barrier to excellence. In the typical OECD country about a quarter of top performers in science come from a socio-economic background below their country’s average…
There is no crisis in our education system, other than the imaginary one dreamed up by a narrow minded and simplistic Minister of Education (a transient position for any incumbent, as opposed to a lifetime vocation for those directly involved) as the comparative tables in reports such as these clearly show.
In fact, for achievement in relation to the per capita education spend, New Zealand does extraordinarily well and we should all be proud of that. That doesn’t mean we should rest on our laurels, but it does mean we should especially listen to those who have made this possible – the teachers.
August 23rd, 2010 at 11:18 pm
@Dave: Also, student performance would be given in terms of the individual student for the parent and in terms of the class members for the aggregate. Hell, if there was enough disparity between groups of students in the class you could potentially work out why – if you had enough information about the students (different types of learners, etc) – and adjust for that, after some time, assigning students to fit teachers to some degree, or giving teachers profiles of students in order that they can best help them learn.
August 23rd, 2010 at 11:20 pm
@Luc: If there is no crisis, and I am not aware that there is, what is the harm in attempting to get better measurement of student and teacher results?
August 23rd, 2010 at 11:30 pm
Also, is it the teachers who oppose this, or is it the union? It seems it would be in the union’s interest to oppose this, as it is likely to take power away from it, and possibly teachers who are mediocre to poor, but not necessarily the majority of teachers. Many may oppose it on principle, I suppose – change is to be feared after all (or many have the instinct to fear it anyway) – or because the union has convinced them it is in their interests to do so, but is this really contrary to teachers interests? If so, what is so bad about it?
Performance measurement is not uncommon. So I guess it begs the question, what kind of measurement of performance already takes place? Is more necessary, or if not necessary would more or a different kind of performance measurement be better?
August 23rd, 2010 at 11:39 pm
“If Parliament used the PPTA system, who would be the best-paid MP…
George Hawkins”
Hehe, John you forget Alamien Kopu, that wonder brought to us by the “Alliance”, me, I thought she was the cleaner. Did we ever get the missing office equipment back? But your point is well made just the same.
August 23rd, 2010 at 11:54 pm
Jivekitty:
I wrote
If standardised tests gives better overall achievement, then I’m all for them. Problem is, all the credible international research shows they are a problem, not a solution.
As for your conspiracy theory nonsense about teachers and their unions, sure, that’s why NCEA, with all its new demands was instituted enthusiastically, and why primary schools are in the process of adopting a new curriculum – because they are fearful, opposed to anything new, don’t want performance measures blah blah blah (the latter true of some, for sure, and that’s the bosses ie principals job to sort out)
The performance review that takes place is through the principals – this group is answerable for that. Teachers can be, in fact, awarded management units with very little actual management duties as a performance reward. That’s the way it works at the moment.
No reason why it can’t be changed and improved, but that’s not about national standards or the bash-a-teacher-a-day agenda popular on this site and well f
eromented, like Islamophobia, by this site’s owner.August 24th, 2010 at 12:07 am
Many, Luc, many. I didn’t say all. The unions didn’t have much incentive to oppose NCEA – it wasn’t likely to impact overmuch upon their power dynamic. Some form of teacher performance rating in this manner does. It would be self-interest on the part of unions to oppose.
As I’ve noted, I do have problems with this as it appears to stand, and I don’t think all critiques are invalid, but I also don’t think that the idea is an inherently bad thing in and of itself.
August 24th, 2010 at 1:08 am
jivekitty
That’s cool, but what’s your evidence?
Or is evidence not a high priority for you?
August 24th, 2010 at 6:37 am
When kids leave school unable to read or write that’s enough evidence of a failing system, especially after we have spent how much on each teacher to teach those kids.
You dumbed down lefties never learn Luc. Your are just too selfish to look at your lazy selves.
August 24th, 2010 at 6:49 am
If other complex goods and services can be allocated to people via a marketplace then why not education?
If other complex goods and services can be allocated to people via a public service then why not groceries? Turning this into a Glibertarian ideological argument a la James has no point to it.
When kids leave school unable to read or write that’s enough evidence of a failing system, especially after we have spent how much on each teacher to teach those kids.
You know, I think you’ll find the capabilities of children follow a bell curve whether you measure teacher effectiveness or not.
August 24th, 2010 at 8:51 am
Sheesh! There’s glib and there’s facetious (that is, I assume you do know about the failure of the USSR, Cuba, etc)
In any case I did not want to get into some philosophical debate about libertarianism vs statism. I just think our public system of education will increasingly be left behind and we can either move forward with eyes open or fall backwards into it.
In NZ at least my bet would be the latter.
August 24th, 2010 at 9:22 am
My sharp-witted little niece made great progress last year but this year at 7 years old is in ‘booster’ maths (just love these teacher euphemisms for cabbage classes) and hasn’t moved up one reading level all year. Has she got dumber? Or is the reason that her teacher, the Decile 9 school’s deputy principal, is in the classroom only one day a week and the class is taught by THREE relievers over the other days?
The big scandal in NZ is primary schooling which fails utterly to teach all our kids to an adequate level of literacy over EIGHT long years!
Warwick Elley’s maunderings could have been written at any time over the last 40 years on any new assessment initiative – and in fact were – by him. I’d love a few more teachers teaching to the test and then more of our kids might pass!
August 24th, 2010 at 9:34 am
There has been quite a response to the LA Times article that DPF quoted but here is an interesting comment (and recaps some of what I have been talking about) but the National Academy of Science is a much more authoriative source.
From “John Rogers Associate professor, UCLA Graduate School of Education and Information Studies and director of the Institute for Democracy, Education and Access”
said at the LA times site…
The National Academy of Sciences has identified several of the problems posed by value-added methods.
First, the National Academy of Science notes that student assignments to schools and classrooms are rarely random. It’s not possible to definitively determine whether higher or lower student test scores result from teacher effectiveness or are an artifact of how students are distributed.
Second, you can’t compare the growth of struggling students with the growth of high performers. In technical terms, standardized tests do not form equal interval scales. Enabling students to move from the 20th percentile to the 30th is not the same as helping students move from the 80th to the 90th percentile.
Third, estimates of teacher effectiveness can range widely from year to year. In recent studies, 10% to 15% of teachers in the lowest category of effectiveness one year moved to the highest category the following year ,while 10% to 15% of teachers in the highest category fell to the lowest tier.
The National Academy of Sciences concluded that value-added methods “should not be used as the sole or primary basis for making operational decisions because the extent to which the measures reflect the contribution of teachers themselves, rather than other factors, is not understood.”
August 24th, 2010 at 9:57 am
According to the PPTA, if we paid teachers another 1.5% (they are asking for 4% and have been offered 2.5%) the future of the country will be assured. What a great investment! If high school teachers need only 1.5% to make them work harder and better and guarantee fantastic results let’s give it to them. And then make sure they fulfil the claims as stated in the full page ads their well-financed union churns out.
August 24th, 2010 at 10:04 am
Kimble said
K> MP, how is it a trick question?
K> If failing the exam, as determined by not getting half the answers right (a low hurdle rate IMO, increase that # to 21 instead of
K> 20 and see how much the probability drops by), indicates that a student has not mastered the course, and if the chances of
K> selecting enough correct answers by chance is ridiculously low, then surely the probability of passing a student who guessed his
K> way to passing the exam is also quite low. Right?
(1/4)^40.
However, if your kid got 31% by hard work and another kid got 31% by guessing and a bit of luck, would you think multi-choice tests are that great?
The point is: we shouldn’t be judging a teacher’s performance by such a flawed method of testing student achievement.
If we are going to test teachers by how well teachers do (and make their income depend on it) then we should be getting an accurate and credible measure of student achievement.
MP>> “Really! I think there is going to be a lot more ways of doing it wrong then the three options available.”
K> True, but it can identify SPECIFIC problems, which is what I said. Addressing specific, or common, mistakes is the low hanging
K> fruit. Pick them up early, correct them, retest, job done. When teaching a class, knowing specificly which techniques to reteach
K> to gain the greatest advance in understanding is very useful
And my original point was that multi-choice tests couldn’t pick up all the types of mistakes that children make including incomprehension and misreading. So I think we’ve got to the same point.
MP>> “That still doesn’t get around comprehension issues.”
K> And like I also said, comprehension is tested in English tests, not maths.
Comprehension is vitally important in written maths tests where the student has to read the question and understand it before they can answer it.
In America there is some complaint that the SAT math exam doesn’t get the best maths students into the best colleges because the level of language comphrenension is to difficult in the maths exam.
MP> “In the American system, IIRC it’s about 40 secs a question so mucking about if you work it out wrong is very costly in terms of answering the remaining questions.”
K> Maybe, but at least they have the opportunity to do it. It is instant feedback, and feedback is good in education.
The biggest lesson that kids get from finding their answer doesn’t match any provided in a testing situation is that tests are stressfull and that they are not good at them.
MP> “but noone should be fooled into thinking they are testing more than a very narrow range of ability or that they are a learning tool – unless there is reasonable feedback at each answer point.”
K> Testing a very narrow range of ability is good, in elementary mathematics. The assumption is that these are on-going tests through out the year, not the final exams, and that reasonable feedback is provided.
A lot of schools are ditching homework because by the time the homework is returned to school, marked and given back to the student the opportunity from learning from any mistakes has been lost. Young kids (<Y6) need feedback close to the time of the mistake because getting it out of context doesn't help them learn anything. I don't see that it's any different from exams and tests.
August 24th, 2010 at 11:06 am
http://www.learningfirst.org/la-times-goes-astray
http://www.learningfirst.org/will-la-times-story-have-chilling-effect-education-research
- read these to find out why the “mediaisation” of education is bad news for student achievement levels
August 24th, 2010 at 11:40 am
@Luc: My evidence for thinking it’s not an inherently bad idea, and that its quality – good or bad – will be in implementation? I’d like to see a cost/benefit analysis. *sigh* You’ve seen my ideas for what I’d try to implement – at least in terms of information available. Whether it would meet a cost/benefit analysis, I don’t know, but I do think that if information gathering is going to occur, then most of the information I’d like to see will be available, it’ll just be a matter of collation – and a decent statistical program should be able to do that. However, from what I’ve seen nothing close to the information I’d like to see will be available?
August 24th, 2010 at 11:44 am
At least collated and available to be seen.
August 24th, 2010 at 11:50 am
@Jane – the world is changing. Go back 150 years and there were no cars, no aircraft, no tv, no tadio, no internet etc … and yet a classroom would look and operate pretty much the same was as it does today, not withstand a few pedagogical improvements. Education is set for a fundamental shake up, and organisations like the NZEI can get on board, or put their heads in the sand and pretend it’s still 1860. Oh, and welcome to kiwiblog!
August 24th, 2010 at 2:39 pm
Just in case you missed it.
DPF said: I think it is more important that a teacher raises someone from say the 40th to the 55th percentile than from the 85th to the 100th
Why????
And, even so, the statistic for “value added” (VAM) that was used doesn’t descriminate between the two teachers in your example.
The teaching skills for those two teachers is going to be quite different and that’s not going to be captured in the VAM either.
August 24th, 2010 at 3:20 pm
Jane Blaikie – visited the links you posted. They are opinion pieces, and everyone is entitled to free speech. However to me they both come across as “we don’t like what you say and if you’re not careful about what you say we’ll shut you out of the schools”.
I appreciate that as a group teachers are wary that statistics can be misused against them, but as a parent I’m not against teachers unless they’re not doing a good job.
If measures can be implemented to help identify teachers that need extra training (and if they’re actually irredeemable then yes, they face the sack just like I do in my job) then I see that as a benefit for NZ kids. I also think if teachers have an opportunity to increase their income through teaching more effectively then where is the downside? The tricky part is calculating a good method to determine their effectiveness – and that’s not impossible (come on, some teachers must be good at maths and statistics still, mine were).
BTW, the Learningfirst.org site lists among their members “The American Federation of Teachers (AFT) is a 950,000-member union of public and professional employees…” – not exactly an unbiased source.
August 24th, 2010 at 3:25 pm
mpledger – you better not be a maths teacher:
DPF said: I think it is more important that a teacher raises someone from say the 40th to the 55th percentile than from the 85th to the 100th
Increase from 40 to 55 = 37.5%. Increase from 85 to 100 = 17.6%. That’s the simple (mathematical) reason.
The obvious reason for me though, is you are helping someone who is failing at the subject (at 40%) and bringing them up to a passing grade level (at 55%). Of course, in my day School Certificate was the measurement, I’m not sure if a score of 55% has any significant meaning in today’s schools. The point is a student already at the 85th percentile is not having much trouble with that subject. The student at the 40th percentile clearly is, and getting them up to the 55th percentile is a significant improvement for them (see above = 37.5%)
August 24th, 2010 at 6:14 pm
@RigthNow, I suspect the downside would be a neutering (um, spaying!) of NZEI’s power. As a Victorian-era union, they believe their advocacy alone should lead to better pay for teachers, thereby justifying their existence. All this talk of pay for performance is highly unnerving for them.
August 25th, 2010 at 9:51 am
Mpledger says
RightNow (1,219) Says:
RN> mpledger – you better not be a maths teacher:
Nope. I don’t teach anything.
MP>> DPF said: I think it is more important that a teacher raises someone from say the 40th to the 55th percentile than from the 85th to the 100th
RN> Increase from 40 to 55 = 37.5%. Increase from 85 to 100 = 17.6%. That’s the simple (mathematical) reason.
*BUT* that’s not how the VAM statistics (the statistic in the article that DPF praised in his blog piece) works out the achievement of teachers. All you have shown is that some other statistic makes better sense. And that was my original point – that the VAM isn’t sensible.
Notwithstanding that….the students who got a perfect score never had their performance accurately measured. If the test had more and tougher questions they may still have got 100% and the 40%ers gone down to 30%.
If you can’t accurately measure student achievement then you shouldn’t use a flawed statistic to measure teacher achievement.
RN> The obvious reason for me though, is you are helping someone who is failing at the subject (at 40%)
RN> and bringing them up to a passing grade level (at 55%). Of course, in my day School
RN> Certificate was the measurement, I’m not sure if a score of 55% has any
RN> significant meaning in today’s schools. The point is a student already at the
RN> 85th percentile is not having much trouble with that subject. The student
RN> at the 40th percentile clearly is, and getting them up to the 55th percentile is
RN> a significant improvement for them (see above = 37.5%)
Some of those 40%ers may have real problems understanding the work. Some may just need the threat of detention to make them get down to work, get their homework done and put in the effort.
Those operating at the 85% level are already putting in a pretty good effort and motivating them to do better and give them work to match their ability is quite a significant skill.
There are quite different skills and knowledge base needed to get the two groups to move.
Nowithstanding all that, it is vitally important to inspire groups of people to high academic achievement. These are the people who go on to be engineers and doctors – we want to give them the tools and knowledge to help solve some of the worlds pressing problems.