Why kids should grade teachers

October 9th, 2012 at 10:00 am by David Farrar

A great article at The Atlantic.

Nubia Baptiste had spent some 665 days at her Washington, D.C., public school by the time she walked into second period on March 27, 2012. She was an authority on McKinley Technology High School. She knew which security guards to befriend and where to hide out to skip class (try the bleachers). She knew which teachers stayed late to write college recommendation letters for students; she knew which ones patrolled the halls like guards in a prison yard, barking at kids to disperse.

If someone had asked, she could have revealed things about her school that no adult could have known. Once Nubia got talking, she had plenty to say. But until that morning of her senior spring, no one had ever asked.

She sat down at her desk and pulled her long, neat dreadlocks behind her shoulders. Then her teacher passed out a form. Must be another standardized test, Nubia figured, to be finished and forgotten. She picked up her pencil. By senior year, it was a reflex. The only sound was the hum of the air conditioning.

Teachers in the hallway treat me with respect, even if they don’t know me.

Well, this was different. She chose an answer from a list:Sometimes.

This class feels like a happy family.

She arched an eyebrow. Was this a joke? Totally untrue.

In towns around the country this past school year, a quarter-­million students took a special survey designed to capture what they thought of their teachers and their classroom culture. Unlike the vast majority of surveys in human history, this one had been carefully field-tested. That research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most—and least—effective teachers.

This does not surprise me at all. I know from my own experience that most kids at school absolutely know who are the teachers who inspire you and make you want to learn, and those who are ineffective. This is not always the same as who the popular ones are. My chemistry teacher was widely mocked as a robot, but everyone said he was a very good teacher.

The point was so obvious, it was almost embarrassing. Kids stared at their teachers for hundreds of hours a year, which might explain their expertise. Their survey answers, it turned out, were more reliable than any other known measure of teacher performance—­including classroom observations and student test-score growth. All of which raised an uncomfortable new question: Should teachers be paid, trained, or dismissed based in part on what children say about them?

I wouldn’t go that far, but I think student evaluations should be routine.

So far, this revolution has been loud but unsatisfying. Most teachers do not consider test-score data a fair measure of what students have learned. Complex algorithms that adjust for students’ income and race have made test-score assessments more fair—but are widely resented, contested, or misunderstood by teachers.

So this is what the NZEI and PPTA should propose as an alternative – student evaluations.

A decade ago, a Harvard economist named Ronald Ferguson went to Ohio to help a small school district figure out why black kids did worse on tests than white kids. He did all kinds of things to analyze the schoolchildren in Shaker Heights, a Cleveland suburb. Maybe because he’d grown up in the area, or maybe because he is African American himself, he suspected that important forces were at work in the classroom that teachers could not see.

So eventually Ferguson gave the kids in Shaker Heights a survey—not about their entire school, but about their specific classrooms. The results were counterintuitive. The same group of kids answered differently from one classroom to the next, but the differences didn’t have as much to do with race as he’d expected; in fact, black students and white students largely agreed.

The variance had to do with the teachers. In one classroom, kids said they worked hard, paid attention, and corrected their mistakes; they liked being there, and they believed that the teacher cared about them. In the next classroom, the very same kids reported that the teacher had trouble explaining things and didn’t notice when students failed to understand a lesson.

The Hattie research confirms this also.

But Kane also wanted to include student perceptions. So he thought of Ferguson’s survey, which he’d heard about at Harvard. With Ferguson’s help, Kane and his colleagues gave an abbreviated version of the survey to the tens of thousands of students in the research study—and compared the results with test scores and other measures of effectiveness. The responses did indeed help predict which classes would have the most test-score improvement at the end of the year. In math, for example, the teachers rated most highly by students delivered the equivalent of about six more months of learning than teachers with the lowest ratings. (By comparison, teachers who get a master’s degree—one of the few ways to earn a pay raise in most schools —delivered about one more month of learning per year than teachers without one.)

Students were better than trained adult observers at evaluating teachers. This wasn’t because they were smarter but because they had months to form an opinion, as opposed to 30 minutes. And there were dozens of them, as opposed to a single principal. Even if one kid had a grudge against a teacher or just blew off the survey, his response alone couldn’t sway the average.

Student evaluation shouldn’t be the only data a school collects, but it should be a near mandatory one.

Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:

1. Students in this class treat the teacher with respect.

2. My classmates behave the way my teacher wants them to.

3. Our class stays busy and doesn’t waste time.

4. In this class, we learn a lot almost every day.

5. In this class, we learn to correct our mistakes.

When Ferguson and Kane shared these five statements at conferences, teachers were surprised. They had typically thought it most important to care about kids, but what mattered more, according to the study, was whether teachers had control over the classroom and made it a challenging place to be. As most of us remember from our own school days, those two conditions did not always coexist: some teachers had high levels of control, but low levels of rigor.

Again, this meshes with my experience.

No one knows whether the survey data will become less reliable as the stakes rise. (Memphis schools are currently studying their surveys to check for such distortions, with results expected later this year.) Kane thinks surveys should count for 20 to 30 percent of a teacher’s evaluations—enough for teachers and principals to take them seriously, but not enough to motivate teachers to pander to students or to cheat by, say, pressuring students to answer in a certain way.

This would be an excellent Budget 2013 initiative!

Tags: ,

32 Responses to “Why kids should grade teachers”

  1. iMP (2,364 comments) says:

    As a teacher, don’t agree wit this. At that age, kids tend to grade teachers according to how popular they are. A more demanding (better) teacher gets a worse rate than a teacher who lets the kids do what they want and cruise. It’s like asking Turkey’s to rate Christmas.

    [DPF: I suggest you read the entire article, because you find in fact they do not. The student ratings correlate well to achievement]

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  2. chrisw76 (85 comments) says:

    I don’t know if I necessarily agree with iMP, but with teenagers you also have to look out for bullying behaviour where more than one may collude to “get back” at someone else including a teacher. Unless carefully controlled student evaluations could be abused in this way, but on average I could see it as being a positive.

    @ David: Found this article the other day on Team Compensation that talks to some of the difficulties incentives create and how strongly demotivating they can be. I believe this is applicable to the teaching environment as well. http://www.leanessays.com/search/label/Incentives

    Cheers, Chris W.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  3. Kleva Kiwi (289 comments) says:

    I don’t think you and I read the same article IMP. Look at the example questions. The questions asked are not ‘is your teacher popular”, they ask questions that reflex performance and improvement in a classroom.
    Your response is what has become the typical expectation of a union driven body.
    The public wants teachers to be held responsible for the teaching ability. The sooner the better.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  4. iMP (2,364 comments) says:

    I largely agree with the article, except they don’t seem to understand that teenagers do not cognitively think like adults. They rate things differently. Popularity and ‘niceness’ often translate in to “quality” in teenage minds. Most teenagers would, for example, think McDonalds was awesome because everyone eats there, and beautiful people have more value than Ugloids. This would distort the results, regardless of how the Q.s were framed.

    That being sad, Teachers totally need rating, but with a big range of measures (perhaps incl. teen appraisal) but most of the Teachers I know would not rate teen appraisal. Teaching should not be a personality contest.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  5. lastmanstanding (1,281 comments) says:

    Yep. All those many years ago at college I can still remember the good the bad and the ugly. The teachers who could hold the class attention with us hanging on every word and wanting to do well.

    And the others like poor ole Clive Beaumont a French teacher fresh old of uni and training college. Poor old Clive ( this was 1965). One trick was to screw up a small piece of wet paper and use the ball point pen as a blow pipe and pepper the black board as Clive tired to write on it.

    Yep. Kid know which teachers are good and which are bad.

    And thats why the Unions are scared shitless that the kids get to rate their teachers. Cause then the Union and the teachers would have none to hide behind. The tide would go out and those without bathing suits would be exposed.

    Bring it on.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  6. MD (62 comments) says:

    If you are in private enterprise you most definitely are rated by your customers, you thrive or fail directly based on their evaluations. I don’t get to complain that my customers aren’t entirely rational and therefore I should disregard their views, they’ve already taken their money down the road. In this case, the students aren’t the only customer, the parents, and the government are also involved and one of those two actually foot the bill. So a 3 way evaluation split between student evaluations, parent evaluations (based on their assessment of how their child is doing) and the government (if they are funding it) based on test results should be the norm. Of course simply removing zoning and allowing funding to travel with the student would address most of this by allowing parents and students choice and make it apparent which schools are failing.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  7. rolla_fxgt (311 comments) says:

    Brilliant article. Intuatively it makes sense. And the fact they even tested a simplified version on kindergarten kids and it still was accurate, and useful, proves in my mind that it needs to be used in NZ, as part of the assesment of NZ teachers.

    I’ve emailed a number of MP’s on just this matter, yet none have said it can be done, some haven’t even bothered to reply. And some have seemed to misunderstand the issue completely, and answer another question.

    I rated all my uni lecturers, and they got paid based on the results, why shouldn’t teachers?

    Its not hard to get and use a well designed test to measure the teaching ability of teachers. And as this article says, it is far cheaper than other, poorer methods of measurement, such as testing, or peer or professional observations. Mainly due to the fact the kids have a long period of time over which to judge the teachers ability.

    And for those that say that students might be biased in their results, well its no more so than other teachers, or principals, or even other education “experts”. Why are they acceptable, but students aren’t?
    Isn’t it a bit off to say you don’t trust your students, yet you expect them to trust you. Why should they trust you, if you have no trust in them, or their abilities?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  8. Tom Jackson (2,553 comments) says:

    Inmates in charge of the asylum.

    I teach at a university. I get really good evaluations, and students tell me they like my lectures. But the evaluations don’t really tell you much about student learning.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  9. Red Sam (122 comments) says:

    Not uncommon in universities at the end of a course. Sites like Ratemyteacher already exist, but are a chance to rant.

    Often at the end of a term, I ask students to write a PMI (plus, minus and interesting) about the classroom, their learning, and my teaching. I enjoy reading their evaluations. The NZCER often ask students in schools to complete an engagement survey. My school has participated in this a couple of times. The results are very enlightening.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  10. Cato (1,095 comments) says:

    This is high-proof nonesense. We really need to abandon this cult of youth – including youth poltics but also the idea of child management. The primary characteristics of children and teenagers are ignorance and barbarism – the traits that good parenting and schooling are supposed to eliminate. That is not helped by undermining authority figures – including rubbish teachers.

    Should teachers be held accountable and should poor teachers be sacked? Absolutely. But that should be based upon the judgments made by parents and principals.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  11. nasska (11,277 comments) says:

    Good idea but who gets the results of the pupils’ survey? If they could be assessed independently they would be valuable information. If the answers stay within the education system to be cherry picked & the data massaged by union hacks nothing will improve.

    We are not dealing with real people who live in the real world here….teachers consider that all their members strive to do their best & all are equal.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  12. freedom101 (496 comments) says:

    Great post DPF. I would like to see the government moving ahead with this sort of work. Expand charter schools as rapidly as possible, and give parents choice generally. Information is important, and student ratings of teachers is extremely valuable information. Rate My Teacher is a site which attempts to achieve this, but it’s not scientific. Nevertheless, there is still a wide variation in teacher ratings. Take for example, at random, one teacher from Wellington College, who has a very high ranking: http://nz.ratemyteachers.com/craig-blacklock/1360-t

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  13. Rightandleft (663 comments) says:

    Actually student evaluations of teachers are already a compulsory part of most schools’ appraisal processes for teachers. I have to be rated by two classes I teach every year, with the results put directly online and sent to my Head of Department for analysis. At my school there was a case not long ago where a class together wrote a letter to the principal asking for a new teacher in a certain subject because their current one was not letting them learn. That teacher was immediately placed on competency review and fired at the end of the year. Students already have a say in teacher evaluations.

    However their reviews are just one part of the evaluation, along with class observations, test scores and self-review against the teaching standards. Showing you have taken part in professional development and contributed to extra-curricular life is also important. Teachers cannot progress to the next pay level without their HOD and the principal signing off on these reviews annually. Thus pay is already linked to performance.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  14. barry (1,317 comments) says:

    Oh David – do really expect the teachers unions and the principals associations etc to go along with this !!!!! The unions and the principals dont want change. They want to keep the status quo – which will continue to produce 25% of their students as effective welfare fodder (and mostly labour voters – if they vote)

    Youd have to have a very careful student assesment because whats just happened with Alan Jones in Ausy (by mostly young people and their social media) is a perfect example of how things are so wrong (Since the boycott of his radio station, the cost of advertising on the his station has RISEN – because despite what the youngies think – most people in Ausy actually agree with jones – Gillards father ceratinly sufferred from shame due to hi daughters lies. The stations ratings have gone UP and those dopey advertisers who withdrew are going to have to pay more to go back….)

    I certainly agree with teacher ratings and teacher pay – and the only reliable method so far is the class progress through the year. ie: the teacher is supposed to lift the class ratings by a year in a year. It doesnt matter how low or high they start – its all up to that teachers ability to get the class to progress.
    Belive eme – it will come – and it will come via National Standards. Everyone has heard the teachers say they are unreliale and un moderated – well wait till they are moderated – by class. Then the shit will hit the fan when parents see that their child has a teacher who cant make the expected progress. theyll demand class changes or a new teacher…….

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  15. Mark (1,480 comments) says:

    Youd have to have a very careful student assesment because whats just happened with Alan Jones in Ausy (by mostly young people and their social media) is a perfect example of how things are so wrong (Since the boycott of his radio station, the cost of advertising on the his station has RISEN – because despite what the youngies think – most people in Ausy actually agree with jones – Gillards father ceratinly sufferred from shame due to hi daughters lies.

    You have got to be fucking kidding trying to justify what an arse Jones made of himself over his comments about Julia Gillard’s father. So now it is social media’s fault that Jones made a dick of himself. You have been reading too many of Jones’ media releases.

    Correct me if I am wrong here, you are also suggesting that students moderate National standards results and that should be the basis for performance pay for teachers.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  16. RRM (9,834 comments) says:

    Seems like a good idea as long as the student reviews are weighted appropriately – I would expect a lot more malicious (and mischievous) reviews from a bunch of school pupils than (say) university students doing the teacher surveys…

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  17. Mark (1,480 comments) says:

    Interesting article. My son’s opinion of his teacher does tend to vary depending on whether he has done his homework and handed it in on time but that aside the survey say at the end of the year has some merit.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  18. RRM (9,834 comments) says:

    How many new public sector employees would be needed to issue, read, collate and report on all these surveys?

    Or will current MoE staff be expected to just do this stuff as well as what they’re already doing?

    (If the latter, will we be condemning them on here when they – inevitably – start talking industrial action because they haven’t had a pay raise in 4 years despite increases in workload and decreases in the quality of conditions?)

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  19. nasska (11,277 comments) says:

    …” will we be condemning them on here when they – inevitably – start talking industrial action”…..

    Of course we will…..after all there’re only a bunch of useless ex teachers who suffered from stage fright when they had to stand up in front of a class. With any sort of luck they might resign en masse at the thought of doing a bit of work. Then the Ministry may be able to hire some non socialists willing & capable of supporting the person unfortunate enough to be a National Minister of Education.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  20. Rightandleft (663 comments) says:

    There is a potential danger for student surveys being less useful if the students know they are being used to determine teachers’ salaries. At the moment we tell them the surveys are to help us teach better. If they thought the surveys gave them real power over their teacher’s salary and job security there would be more potential for issues.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  21. nasska (11,277 comments) says:

    Rightandleft

    …”Teachers cannot progress to the next pay level without their HOD and the principal signing off on these reviews annually.”…..

    Genuine question. Has this ever happened?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  22. Rightandleft (663 comments) says:

    Do teachers not progress to another pay level? Yes that does happen and generally it also starts immediate competency procedures. You won’t hear about competency procedures in the media because they are not the same as teacher registration involving the Teachers Council. Teachers found to be incompetent and dismissed are not stripped of their registration, though they will not be able to renew it when it expires after three years. However in the current job market it is highly unlikely anyone dismissed for incompetency will be hired by another school.

    And as a direct example, the teacher dismissed from my school that I mentioned. That teacher was refused their pay increase the year before based on a bad appraisal by the HOD. The principal would rarely be involved in individual teacher appraisals, they have to take the word of HODs over who is performing and who is not.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  23. nasska (11,277 comments) says:

    Rightandleft

    Thanks for the response. Would it be fair to say that there would be a reasonably long drawn out procedure to be followed between the teacher being turned down for a pay rise & being dismissed? If so it would seem that the education of thirty odd kids each year would be well & truly compromised.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  24. BeaB (2,118 comments) says:

    IMP
    In my long experience. kids would always choose quality teaching over popularity, even deserting the most popular teacher in the school (for jokes, fun, diversions etc) for the nerdy, humourless one who gave them good notes, solid lessons and thorough prep for exams.

    Kids are no different from us – they know who is good at their job and who makes sure they get their work done. Just watch them going into one class taking off their earrings and pulling up their socks and slouching into another. John Graham used to say if he told the boys they could choose to be taught by the best English teacher in the school there would be a stampede to one classroom.

    Kids aren’t the only ones who can discriminate between the good and the mediocre. Other teachers can too as can the principal. The problem is then what you can do about it as outright incompetence is easier to prove than mediocrity.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  25. Rab McDowell (6 comments) says:

    I serve on the Board of Trustees of a secondary school. Boards have a duty to appraise staff performance. Normally they delegate the senior management to do this on their behalf.
    One year I suggested, not altogether tongue in cheek, that we we require all students to rate their teachers on Ratemyteacher.com for the very reasons you outline in this article. You will not be surprised to find the staff did not welcome my suggestion. Needless to say, I did not get much support.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  26. mpledger (425 comments) says:

    Kids don’t have enough experience of teachers to know who is a good teacher. All they can do is mark relative to the few teachers they have had. And I have to say before year 4 (which is before they start testing kids with written tests) the answers will be pretty much a nonsense.

    Since Felix Salmon can do a better job than I can tearing this idea to smithereens go see….
    http://blogs.reuters.com/felix-salmon/2012/09/23/what-education-reformers-did-with-student-surveys/

    It’s another beuracratic waste of money. More time wasting instead of time learning.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  27. mpledger (425 comments) says:

    Hey, what about dicking Mark. His comment about Julia Gilliard’s father seem just as inappropriate as the people who got dicked for saying the same things about whale oils’ mother.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  28. wf (428 comments) says:

    Poster #1 is, I gather, a teacher. He woud serve his purpose more effectively if he proof read his posts before submitting them.

    My grandson who is 14, told me that his class teacher was a nice guy, but he didn’t learn much from him, but it would be ok because next year he would have Mr X and he’d catch up. I commented that it seemed like a wasted year, and got a shrug of the shoulders in agreement.

    And kids of today are a different breed than those of a generation back. They are surrounded by trash which ever way they turn and learn to make value judgements very quickly – especially when information turns out to be worthless or inaccurate.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  29. slijmbal (1,230 comments) says:

    mpledger says “Kids don’t have enough experience of teachers to know who is a good teacher”

    and there we disagree – in my case based on personal experience.

    At my school we knew teacher X meant failing French O Level if you didn’t catch up outside of his class for instance. Our Engish Lit teacher was an ex army man who behaved like we were in his little army – behave and do what you are told – mind you we all passed O Level English Lit – the entire class. The school knew who were the better teachers as there was streaming and the better teachers taught the better performing children (in hindsight they might have been better switching that around as the better performing children tended to manage around poorer teachers).

    On University lecturers – by the time you get to Uni you’re pretty much on your own I found – most lecturers were awful at teaching as most were really there to research and there is a totally different set of skills required for research and teaching. However, by this stage you really should be receiving direction and assistance rather than being taught – it’s a more adult approach. One really should not need to be spoon fed learning by this stage.

    I did pure science so would expect (hope?) this to by quite different in more vocational courses.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  30. Rightandleft (663 comments) says:

    Nasska, I don’t think the procedure is any more difficult than for any profession. Teachers can be put on competency procedures for even a single parent complaint. Even the principal walking past the classroom and thinking it sounds too loud and out of control could be enough to initiate procedures. However the goal of competency is not simply to fire a useless teacher it is to upskill them and bring them into line with the standards. They are given time to prove they can improve and only dismissed if they fail to do so. They will be observed regularly throughout and it is on their shoulders to prove they have corrected their errors. In the case I cited the time between the complaint being made and the teacher being dismissed was about ten weeks or one term.

    One problem with blaming bad teachers for everything is that more than a few are being forced to teach out of subject. I know that social studies is often treated as a subject anyone can teach when that simply isn’t the case. As a result I have seen PE, English and Careers teachers struggling to teach the subject and no doubt causing students to go home convinced that teacher is useless, when in fact they are a fine teacher in their actual subject area.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  31. MH (711 comments) says:

    Teachers are female,there’s the problem. Solve that solve the world. Did this study mention the sex of the teachers?

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  32. HB (319 comments) says:

    ummm…..
    more than half the teachers at the school I teach at are male.
    Not sure why females are a problem that needs solving?

    I can also confirm what rightandleft says above about teachers being accountable.
    We also had a teacher at our school who no longer teaches. This was not due to a parent or student complaint but came to the schools attention during the appraisal process.
    It is also compulsory at our school to get student feedback about our teaching. We also survey about many other things such as bullying. It is a common occurrence for senior management (including the principal) to ‘walk through’ our classrooms for the express purpose of seeing what is going on. It is usual for them to talk to students during these ‘walk throughs’ ”what are you learning about?” “why are you learning about this?” (or similar) are pretty revealing.

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote

Leave a Reply

You must be logged in to post a comment.