Thursday, September 22, 2011

Data vs. judgment

Gary Gutting has some sanity -- “critical thinking,” even! -- about the limitations of standardized test score data. This passage in particular caught my attention:
There is also the question of whether any standardized test is adequate or needed to evaluate certain sorts of student learning. There was a time when we were happy with Miss Goodteacher’s judgment that her class knew how to read. There are doubtless cases where we can’t trust instructors’ judgments. But is there reason, especially in college-level work, to think that this is generally the case?
Sometimes, when I complain about the effect that high-stakes testing is having on K-12 education, I get asked: “But how else can we tell if students are learning, or if teachers and schools are performing well?” I’m never sure what to say first in response to that question. Part of me wants to say, “Yes, it would be great if there were an objective measure of those things, but wishing doesn’t make it so.” Another part wants to say, “But the benefits of using such an imperfect way to assess those things might be far outweighed by the ill effects of using it -- for example, by the increasingly narrow focus of schools on one or two goals -- ‘teaching to the test’ -- at the expense of all others.”

But usually what I end up saying is: “Hire good people, give them enough pay and enough autonomy that they’ll stick around and develop wisdom and judgment, and then let them use their wisdom and judgment.” Many people seem to find that unthinkable, and would much prefer the false security of a number, regardless of what that number represents and how that assessment distorts the educational process. Yet my answer is a pretty close description of how our higher education system works. It’s also a pretty good description of Finland’s vaunted educational system. Is it so outlandish to think it might work in our K-12 schools?

I think people distrust that approach because they don’t have faith that our K-12 teachers would have the good judgment necessary to make it work. But isn’t that a self-fulfilling prophecy? We rely more on standardized tests, and teachers are forced to teach to those tests, and their teaching itself becomes standardized, and they have less autonomy in the classroom, and their jobs become less satisfying, and good teachers leave the profession, and fewer qualified people become teachers, and so we trust them even less to have good judgment, and so on. Is that a recipe for good education?

It sometimes seems like people are desperate to find some scientific substitute for individual human judgment. But if we can’t count on our educators to have good judgment, how can we count on them to make good use of standardized test data?


Curious said...

After reading through several of your posts, I am just curious...why exactly do you send your children to public schools? It seems as though your tone throughout much of what I have read is that of exteme disbelief in the public school system as a whole. If this is the case, then why not homeschool or send your children to private schools?

Chris said...

Curious -- That's a perfectly understandable question. I think it's probably better answered in a full post rather than a comment, and unfortunately I don't know how much time I'll have for blogging over the next few days, so I'm afraid you'll have to stay tuned until next week . . .

Doris said...

Speaking of the importance of letting teachers use their "wisdom and judgment": Recently a local 5th/6th teacher made the following post on her (public) class website (which I'm hoping it's legal to reproduce here? Chris?):

If I Were the Teacher...... As an initial writing assignment I asked students to compose a brief note on what their classrooms would be like if they were teacher. The suggestions were quite interesting. I've summarized a few here below:

have more say in class

more exercise, PE or recess even if it meant a longer school day

learn another language

more math

more independent work in reading and writing

more help from teachers during recess

develop a science club or an animal/nature club

more field trips

wear uniforms

make our own class rules and follow them

rewards for good behavior like a day to chew gum

For the most part these were ideas and concepts I'd love to work into our daily life at school. Although no one asked for more tests, I think all of us would like to find more ways to demonstrate the learning the students have achieved. In our entertainment saturated culture it is hard to maintain a bright line between what is learning and what is merely entertaining. In my experience aiming for wonder is what it's all about. All of us have had those moments where our curiosity has led us into learning that has been deep and engaging. Sparking that kind of wonder is what makes great students and lifelong learners. As we start the year I hope we can be partners in getting that spark going in all our students in Room 13.

Here's the URL:

My hat is off to this teacher (whom I do not know) for starting off the year by giving her students an opportunity to express their own vision for how the school might be run.

Chris said...

Doris -- Thanks, that is a nice exercise. (Here's the clickable link.) The teacher seems to have a great philosophy on what school should be about.

I'd be in favor of giving the kids a real say over at least some of the classroom rules. Maybe I'm crazy, but I don't think they will choose murderous anarchy. On the other hand, they might decide that recess shouldn't be used as a punishment, and they might have different ideas about how quiet the lunchroom should be . . .

Karen W said...

Montessori involves kids in setting classroom rules. It turns out the kids appreciate order, and fair and respectful treatment, as well as a chance to participate in decision making just like adults do.

Chris said...

Karen W -- Thanks for commenting! Montessori provides a lot of good examples of how it’s possible to do some things differently. Sometimes it seems like no one in our school system is even willing to acknowledge that there is more than one way to think about education. I think that accounts for some of the wrong turns our district has taken.