No price too high for standardized tests?

Cuts in music and foreign languages? “Sorry, times are tight.” Millions for new standardized tests? “Only the best for our kids!”
— Chris Liebig (@Chris_Liebig) January 7, 2015

[This post appears as a guest opinion in the Des Moines Register and Iowa City Press-Citizen; I’m posting the published version here with some additional notes afterward.]

Should we spend three times as much on standardized testing as we currently do, or eight times as much?

That question captures the range of current establishment opinion. The establishment answer, of course, is eight times as much—and more.

Currently, Iowa requires schools to administer the Iowa Assessments every year. Because those tests are developed by the University of Iowa, we get them at cost: about $4.25 per student.

But in 2013, the state created a task force to determine whether to require a different set of assessments as of 2016-17. The task force considered two possibilities: the Next Generation Iowa Assessments and the Smarter Balanced Assessments. The former would cost $15 per student. The latter cost significantly more and would need to be supplemented with a separate science test, for a likely total cost between $30 and $40 per student. Those tests would also require unknown millions to buy and maintain the necessary technology; the task force—incredibly—didn’t even try to estimate the cost.

The tests we’ve used for years are now inadequate, we’re told. We need tests that incorporate “constructed responses” that require human graders, or tests that the students will take on computers, with technology that can adjust the difficulty of the questions to the student’s responses. We need to make the kids sit through twice as many hours of testing as they do now.

The task force recommended the most expensive tests (though not unanimously). If you suspect there are diminishing returns from all this expensive additional testing, you’re a Luddite.

What will get cut to pay for these tests? No one will say. Don’t believe for a minute that the state will fund these new tests at no sacrifice to other educational funding. Every dollar the state spends on testing will be one less dollar available for general school funding—at a time when many districts are already making severe budget cuts. No matter how it’s spun, the school districts are about to get hit with an enormous unfunded mandate.

Standardized testing is now the Defense Department of the school budget: only the most deluxe, big-ticket, exorbitant program will do. Never say no, regardless of what has to be sacrificed. The testing companies get richer; your kids’ education gets poorer.

The only thing standing between us and this enormous increase in spending on standardized testing is our legislature. If your school district has to make class sizes larger, cut band and orchestra, eliminate foreign language classes, or worse—all so we can have the shiniest new standardized tests—you’ll know who’s responsible: state legislators who decided that no price was too high.

________________

Here’s some additional information that I couldn’t fit in the guest opinion:

By not even trying to quantify the costs of the technology that will be necessary, the task force violated the legislature’s explicit charge. The act creating the task force provided that “the task force shall consider the costs to school districts and the state in providing and administering such an assessment and the technical support necessary to implement the assessment.”

The dissent to the report focuses largely on the fact that the task force did not assess the full costs of the Smarter Balanced tests. You can read it in full here.

________________

The task force report works hard to minimize the actual cost of the Smarter Balanced Assessments, stating that the tests will costs about $22.50 per student. In fact, the cost will be significantly higher, for several reasons:

First, the $22.50 figure is an estimate that comes from the Smarter Balanced Assessment Consortium—that is, by the people who have a major interest in getting the state to adopt the tests. In fact, as the Consortium admits, the actual cost will depend on what is charged by the private vendors (Pearson?) that will administer and score the tests. If those charges come in higher than the Consortium’s estimate, no one will be surprised.

Second, the Consortium’s additional “interim” tests add another $4.80 per student to the cost. The report refers to those tests as “optional,” but its inclusion of this chart seems to promote their use. (These interim tests would also add to the number of hours the kids have to sit for tests.)

Third, Iowa law requires a science assessment. Unlike the (far less expensive) Next Generation Iowa Assessments, the Smarter Balanced tests do not include a science assessment, so the state will have to pay for a separate science test. That test could cost another $10 per student.

Fourth, if some districts are not ready to conduct the tests via computer, they can use paper-and-pencil versions of the tests—for $10-12 more per student than the online versions.

For those reasons, I used an estimate of $30 to $40 per student, and that’s just for the tests themselves, not for the necessary technology. I would have preferred not to estimate at all, but the task force didn’t assess those costs itself.

________________

Then there are the technology costs. Even the task force report admits that its estimate of the per student cost does not include the tech costs that will be necessary in districts across the state of Iowa—which, as the dissent points out, are likely to be “significant and ongoing.” What will those costs be? Don’t ask the task force.

The tech costs won’t just be for equipment and bandwidth, either. They will almost certainly require ongoing expenses for tech staff to maintain the tech infrastructure, as well as ongoing upgrades. There goes a teacher.

On the lack of information about the tech-readiness of Iowa’s school districts—and on some of the debacles experienced in other states—read Karen Woltman’s post here.

________________

The task force report argues that using the Smarter Balanced tests could enable districts to save money by eliminating some other tests they are currently using. But if that’s true, it’s also true of the much less expensive Next Generation Iowa Assessments, and the task force doesn’t claim otherwise. Moreover, this argument conveniently ignores the fact that the Smarter Balanced tests do not include the statutorily-required science assessment—which means the Smarter Balanced tests will actually cause expenditures on additional tests.

________________

Though it works hard to minimize the cost of the new tests, the report also works hard to make the current tests look more expensive. Although the Iowa Testing Program lists the cost of the current tests as $3.50 per student, the report says that they cost “$4.25 to $6.25” per student, citing a document that is not available online.

The $4.25 figure supposedly comes from adding a $.50 processing fee and a $.25 barcode fee. I’m willing to trust the report about the existence of those fees, though a source would be nice.

The $6.25 figure is what districts would pay if they didn’t agree to allow some field-testing of new questions—which means no one has to pay it.

Then the report asserts that the current Iowa Assessments have a hidden $2.25 additional per-student cost because of “additional data managing and reporting” costs borne by the state. I didn’t include that figure in my estimate, for three reasons. First, the report gives no citation at all for that assertion. Second, there is no way to tell whether the report is comparing apples with apples: will there be similar “data managing and reporting” costs if we use the Smarter Balanced tests? Third, if the report is just counting money the state spends using the test results, it can’t fairly include that as part of the price of the tests.

Moreover, even if you include the $2.25, you’re only up to $6.50—a far cry from what the Smarter Balanced tests cost.

________________

One of the most striking paragraphs in the entire task force report is worth reprinting here in full:

The other costs that cause concern are related to districts’ technology readiness to support online testing. While there is no doubt that some districts are behind in technology readiness, schools will not be required to make devices and internet connections available to each and every child simultaneously. As the Consortium notes, “A 600-student middle school could test its students using only one 30-computer lab.” In these ways, the costs of upgrading school technology infrastructures are not likely to be overly burdensome on the whole. Besides, these are costs the state of Iowa should shoulder. We must better incorporate technology into the delivery and conduct of not just our assessments, but our instruction as well. The future of assessments is online, but so is the future of teaching and learning. Investing in devices and bandwidth is necessary and should be done by the state regardless.

Well, the Consortium assures us that we can test 600 students with just 30 computers, so we don’t have to bother quantifying technology costs. You might think giving 600 seven-hour-long tests on just 30 computers would be “overly burdensome,” or, say, an unimaginable nightmare. But the salesperson lobbyist Consortium told us it’s okay, so it’s cool.

But anyway, we should buy a lot of technology! We’re sure it will be useful and necessary, for lots of stuff that we can’t actually identify because we’re just an assessment task force. But that’s what the future’s all about, right—computers? So there’s no need to consider how much it will cost, even though the legislature told us to.

Okay?

________________

Don’t even get me started on the assertion that “the future of teaching and learning” is online.

________________

Notice the double standard. When the report wants to make the existing tests look expensive, it includes extra costs that it claims are borne by the state. But when it wants to make the Smarter Balanced tests look less expensive, it argues that “Besides, these are costs the state of Iowa should shoulder”—as if that somehow means they don’t cost the districts anything. Of course, any money the state spends on testing is that much less money it can spend on other educational needs, including supplemental aid to the districts.

________________

Although the task force didn’t quantify the total cost of adopting the new tests, it assures us that this unknown cost is “proportionately small,” compared to the total education budget. So what? It’s also small compared to the national debt, or to the gross national product of Bulgaria. What matters isn’t how much of the total budget it is; what matters is how we’re going to pay for the increase. What will get cut to pay for these tests? If we don’t know, how can we know whether the tests are worth having?

Instead of giving us a cost-benefit analysis, the task force gave us a benefit analysis—really, closer to an advertisement. Is that what the legislature asked it to do?

________________

One reason the Smarter Balanced tests are so expensive is that they use “computer-adaptive testing,” which means that the technology will vary the difficulty of the questions in response to a student’s answers as the test goes along. There is not universal agreement that computer-adaptive testing is even desirable, though, let alone worth paying for. Commenters make some interesting points about it here.

________________

The report emphasizes that computer-based tests will enable districts to get the test scores back much more quickly. Is that worth paying a lot of money for? The tests are going to be given toward the end of the school year. If Susie gets a lower-than-hoped-for “problem solving” score, will the school somehow re-teach that year’s math curriculum to her in the last month of school? How would they even know what to fix?

If there is a connection between the quick score reporting and the improvement of your child’s education, you won’t find it explained in the task force report. Am I wrong to think that, at best, the scores will be used primarily to evaluate curricular choices and recommended teaching strategies, which schools don’t change overnight? (Large bureaucracies aren’t exactly known for being nimble.) If the ultimate point of the scores is to improve the curriculum next year and beyond, what is gained by getting the scores in days instead of weeks?

________________

The report also asserts that the new tests will provide “more precise scores for Iowa students.” I have a lot of concerns about K-12 education, but I have to admit that I have never complained about my kids’ scores on the annual standardized tests not being precise enough. In any event, call me skeptical that any test will ever “precisely” measure a kid’s “academic performance.” (See this post.)

The report also asserts that the tests will measure kids’ “ability to transfer their learning to real-world situations”—by giving them a standardized test in a classroom. Again, call me a skeptic. (See this post.)

The report also asserts that the tests will help our students “compete against students across the country and beyond” in “today’s global economy,” because the same tests have been adopted by “21 states and a U.S. Territory.” Now Iowa can finally prevail in its age-old rivalry with the U.S. Virgin Islands.

On those topics, more in upcoming posts. In the meantime, I can’t resist raising one question: how much do you want to bet that the same kids who do well on the current (inexpensive) tests will also do well on the new (very expensive) ones? If so, then what is actually being measured?

________________

What’s missing from the report is any concrete explanation of how these tests will be used to improve your child’s education. There are all kinds of fun, expensive things you can do with data. Many of them have no meaningful bearing on real life. A marginally more “precise” “total writing” score might be a victory for the test designers, but just how does it make your child’s life better? Through what series of events does that occur?

I came away from the report with the discouraging sense that our state education bureaucracy has really lost its way. Nobody wants to admit that the ability of empirical data to tell us how to turn kids into capable, independent adults is severely limited. Standardized testing has legitimate uses, but it’s not vested with magical powers. It’s almost as if education officials have constructed a reassuring, imaginary universe in which their job is to manipulate abstractions—like in those infernal box-folding questions on the tests—rather than to deal with the messiness of the living, breathing, infinitely varied bodies and minds who arrive in the classrooms every morning.

They can dream. But if they want us to spend millions of dollars, they owe us much more specific, realistic information about costs, benefits, and what will get cut.
.