Tests don’t measure teachers


If we really care about the education of young people in poverty, we will stop focusing on test results and pay much more attention to the quality of life students and families endure. The more their parents and the students themselves are employed, the better their housing and transportation, the better their health care and nutrition, the more they learn.

Propaganda for testing and fear, however, recently got a boost from media coverage of a well-publicized study out of Harvard and Columbia universities. The study centers on “teacher value added” or VA _ teachers’ rankings by improvements in their students’ test scores. The New York Times headlined its report on the study, “Big Study Links Good Teachers to Lasting Gain.” But the researchers conclude that for students coming from poverty, the “lasting gains” are very small when present at all, and are only gains relative to other students in poverty, not relative to wealthier students, who remain far ahead by all measures.

For low-income students, the authors write, “Teacher VA does not have a significant effect on college attendance rates at age 25.” “Not having a significant effect” is hardly a “lasting gain.”

By age 28, the study discovers, low-socioeconomic-status students of elementary and middle school teachers who produced high test scores earned only $2.70 more each week than comparable students whose teachers produced average test scores. In Baltimore, $2.70 is not enough for a round-trip bus ride.

Although the data above come directly from the Harvard/Columbia study, you didn’t see these findings in press accounts because the purpose of the study is to contribute to the propaganda that distracts public attention from the underlying effects of the caste system on education.

Aggregated data help the authors address their main policy concern: whether it makes sense to pay bonuses to “higher quality” teachers, or to “de-select” (their euphemism for “fire”) “lower quality” teachers. They conclude that it makes more sense to fire the lowest 5 percent of teachers because the best teachers are likely to stay teaching even without a bonus. But if the researchers had reported only on students in poverty, they would have had trouble justifying “de-selection” of any teachers at all based on test scores because the effect of “high value-added” teachers as opposed to “low value-added” teachers is either non-existent or very, very small for that population.

It’s interesting, too, that the long-term correlations between the “value-added” measure and students’ lives are inconclusive or non-existent for the great majority of teachers _ the middle 70 percent _ so the authors say nothing about the benefits of replacing, for example, a teacher who ranks in the 30th percentile with one who ranks in the 60th percentile. Their study really only amounts to saying that schools should try to keep the very best teachers and get rid of the very worst _ hardly a deep insight. All of this work is premised, of course, on the dubious idea that test scores are the most important criterion to use in evaluating education.

For comparison, giving employment to teenagers has a much higher lifetime payoff than trying to measure and sort teachers by “value added.” Research shows that students who have a part-time job in high school earn $40 a week more by the time they are 27 than other students. That’s enough to make payments on a car, and it is 15 times the effect the university study found from replacing an average teacher with a great teacher.

The positive effect of attending schools that serve generally better-off students is also vastly more important than teacher quality. According to the Harvard research team, students who go to schools where most students are college-bound (i.e., wealthier students) earned $26,312 a year by age 28, compared to $15,917 for students who go to schools where most students are not college bound (i.e., poorer students). That works out to $200 more each week, 75 times greater than the effect the authors found of improving teacher quality. The authors are completely open, too, about the fact that the positive effects on future earnings are much stronger for richer students than for poorer students; in other words, wealthier parents would be wise to pay more for the best teachers because it will help their kids proportionately more than the same good teachers would help poor kids.

Even aggregating the wealthier and less wealthy students together, the authors found less than half a percent increase in the probability of a student’s going to college if that student had a “high value added” teacher, compared to a student who had a “low value added” teacher. This makes sense: Wealthier children tend to end up in college and poorer children tend not to, regardless of the teachers they had in school.

This study has received prominent attention in the media and is bound to be quoted widely, since it reinforces the “reformers'” prejudice that making teachers fearful for their jobs is a good idea. But a well-educated public wouldn’t buy any of it: “testing” isn’t the same as “education”; the caste system isn’t the same as “equal opportunity”; and “value added” doesn’t make nearly as big an impact on children as supporting, respecting and listening to teachers who care. If you’re not sure who those teachers are, just walk into any school and ask.