Data vs. judgment

Gary Gutting has some sanity -- “critical thinking,” even! -- about the limitations of standardized test score data. This passage in particular caught my attention:

There is also the question of whether any standardized test is adequate or needed to evaluate certain sorts of student learning. There was a time when we were happy with Miss Goodteacher’s judgment that her class knew how to read. There are doubtless cases where we can’t trust instructors’ judgments. But is there reason, especially in college-level work, to think that this is generally the case?

Sometimes, when I complain about the effect that high-stakes testing is having on K-12 education, I get asked: “But how else can we tell if students are learning, or if teachers and schools are performing well?” I’m never sure what to say first in response to that question. Part of me wants to say, “Yes, it would be great if there were an objective measure of those things, but wishing doesn’t make it so.” Another part wants to say, “But the benefits of using such an imperfect way to assess those things might be far outweighed by the ill effects of using it -- for example, by the increasingly narrow focus of schools on one or two goals -- ‘teaching to the test’ -- at the expense of all others.”

But usually what I end up saying is: “Hire good people, give them enough pay and enough autonomy that they’ll stick around and develop wisdom and judgment, and then let them use their wisdom and judgment.” Many people seem to find that unthinkable, and would much prefer the false security of a number, regardless of what that number represents and how that assessment distorts the educational process. Yet my answer is a pretty close description of how our higher education system works. It’s also a pretty good description of Finland’s vaunted educational system. Is it so outlandish to think it might work in our K-12 schools?

I think people distrust that approach because they don’t have faith that our K-12 teachers would have the good judgment necessary to make it work. But isn’t that a self-fulfilling prophecy? We rely more on standardized tests, and teachers are forced to teach to those tests, and their teaching itself becomes standardized, and they have less autonomy in the classroom, and their jobs become less satisfying, and good teachers leave the profession, and fewer qualified people become teachers, and so we trust them even less to have good judgment, and so on. Is that a recipe for good education?

It sometimes seems like people are desperate to find some scientific substitute for individual human judgment. But if we can’t count on our educators to have good judgment, how can we count on them to make good use of standardized test data?
.