They blinded us — with science!

I am always discouraged by how much of the debate over educational policy focuses on factual disagreements to the neglect of value disagreements. Facts are relevant and important, of course, but only in the context of value judgments about what kind of world we want to live in, and what we think it means to be well educated. No amount of empirical evidence can tell us what to want. All the evidence in the world might show that corporal punishment raises standardized test scores, but it doesn’t follow that we should beat our kids.

Somehow, though, virtually all of the discussion of education “reform” proposals is about whether they “work,” without any reflection on what that means. When a study purports to show that a program like KIPP raises its students’ standardized test scores, the program’s detractors chime in to say, “No, it doesn’t!” -- and the entire argument proceeds on the apparently shared assumption that, if they do raise test scores, there can be no argument against them.

Ask these empiricists why we should make raising standardized test scores the only measure -- and thus the only goal -- of education, and their empirical answer is some variation on “Because of course we should.” Even on that rigorously empirical foundation, the inherent weaknesses of social science research render any conclusions extremely contingent. We’re not talking about randomized clinical trials in which one variable is altered while others are held constant. Can the studies of KIPP schools sort out the effects of selection bias and attrition over time? Can they sort out the effects of KIPP’s more authoritarian practices from the simple fact that KIPP provides more child care coverage than ordinary schools? The “empirical findings” of these studies are castles in the air.

None of these questions stop school reform enthusiasts from declaring that “KIPP schools work,” however. The proof apparently goes something like this:

1. Assume, without discussion, that x (increased standardized test scores) causes y (a better adult life).

2. Start a school program that changes thirteen variables at once.

3. Publish data that is consistent with the possibility that your program increases x, but is also consistent with several other plausible hypotheses.

4. Declare that all thirteen variables are proven to increase y.

That’s some weird science! A simpler version of the proof is: “I want to believe it; therefore it’s true.”

Does anyone doubt, though, that if a school is willing to do anything to raise test scores, it can raise them? I suppose we should just count ourselves lucky that the KIPP program didn’t include, say, the use of carefully calibrated electrical shocks. If it had, the studies would obviously prove that such techniques “work,” and any questions about the values underlying the program could justly be ignored.

Having been an Eighties teenager, I must now include this:


.