Saturday, January 31, 2015

Would you hire these people to use technology to teach your kids?

“The future of assessments is online, but so is the future of teaching and learning,” the state assessment task force declared. But our state education department’s use of technology is not exactly an advertisement for technology’s power to inform and instruct.

Yesterday the state Department of Education posted a website designed to rate all of the state’s schools against one another on reading and math test scores. Set aside for the moment whether that’s a good idea; after all, our legislators made them do it. But the site is a mess.

Here’s an example of a graph designed to show the reading proficiency of the kids at my local elementary school:


What are we to make of this graph? How is the green line (labeled “Students Meeting Proficiency”) different from the blue bar (labeled “Meeting Proficiency”)? Why is the green line a line at all? Does the x axis have any meaning whatsoever? If not, what do the dots on the lines mean? And why are the little informational boxes positioned to block the view of the orange and blue bars? (It gets worse.)

Meanwhile, see if you can find your school’s dot on this graph.

Meanwhile, see if you can find Iowa City’s West High School in the site’s drop-down menu. Not so easy.

You might try to figure out what’s going on by clicking on the “More Information” drop-down menu, which has links for “Website Introduction,” “FAQ,” and “Report Definition.” None of the links work.

The site worked even less well on a mobile device than on a desktop.

These are the people who are going to tell us how to improve our kids’ education through technology? These are the people who are going to implement the proposed high-tech (and debacle-prone) Smarter Balanced standardized tests?

More on the rankings here.
.

Wednesday, January 28, 2015

Assess this


This is from New Jersey, but does anyone doubt that it would look the same in Iowa?

But our legislators do love to legislate. Because they know best!
.

Monday, January 26, 2015

School board needs to stop daydreaming about restricting free speech

Here we go again. Our school board is talking yet again about community comment at board meetings. Tomorrow’s agenda includes a Powerpoint presentation (as if board meetings were not sleep-inducing enough already) about all the possible ways to restrict speech by members of the public at public meetings. The agenda item was requested by board member Marla Swesey.

Wasting more time on that topic would be bad enough, but the Powerpoint includes at least one slide that is outright misleading about what the First Amendment permits:


There is absolutely no legal basis for suggesting that the school board could restrict public speakers to “respectful” and “professional” comments. (See this series of posts.) If the board tries it, the district will get sued, will spend money needlessly on litigation, and will lose the lawsuit.

Meanwhile, this was the scene at the Eastside Elementary Orchestra concert at City High last week:


Yes, there were about a hundred people in the balcony, but the main floor, which would normally be crowded, was virtually empty, because fourth grade orchestra was eliminated in the budget cuts that hit the music program (and several other programs) last year. At least one member of the music staff went home after the concert and cried. Now we’re being warned that more cuts are on the way.

To spend one more minute, or risk even one dime, on trying to force community commenters to be more “respectful,” when the district has real problems like that to deal with, is a sign that board members are out of touch with the people they represent.

Kudos to board members Jeff McGinness and Tuyet Dorau, who voted against spending any more time on this futile enterprise. Great post by Mary Murphy on the First Amendment and community comment here..

Is there an optimal squeakiness?

One thing I’ve enjoyed about the book critic James Wood is the way he sometimes soft-pedals his own criticism and lets the book he’s reviewing indict itself. He’ll write a careful, restrained, and reasonable-sounding critique of a book, but then quote passages that reveal the book to be egregiously awful. You walk away from his review thinking not only that the book is terrible but that it’s very decent of Wood not to criticize it more harshly.

Something similar happens, I think, over at Karen W.’s blog. Karen does the unglamorous work of actually reading the education proposals that come out of the legislature and the state Department of Education (so we don’t have to). She comments in a matter-of-fact way and raises a few good questions, but she mostly lets that parade of horribles speak for itself.

For example, you can read her recent posts on the anti-bullying bill that would authorize schools to monitor kids’ social media accounts (and maybe even demand their passwords?), about the state’s plan to rank Iowa’s schools against one another, and about the state’s rules about what counts as “evidence” against its tourism-driven plan to require later school start dates. Even her more extended critiques are written in the calm voice of reason.

Not all bloggers (ahem) can muster that kind of restraint. Maybe that explains how she got invited to be on the state assessment task force, despite her previously skeptical stance toward the Smarter Balanced Assessments. (Or maybe our twenty-first century education officials didn’t realize she had a blog.) She ended up in a minority of one, but her dissent may find an audience with the legislature—certainly more of an audience than a simple blog post would have found.

Meanwhile, over at Parenting is Political, NorthTOmom describes a meeting she and her husband had with the vice principal of their daughters’ school about the amount of homework that the teachers were assigning. NorthTOmom methodically explained to the vice principal how the school was violating the district’s homework policy. When the vice principal refused to acknowledge the problem, NorthTOmom’s husband snapped, “There’s too much fucking homework!” Whether either approach will get results remains to be seen.

Michael Tilley, another local school blogger, recently wondered aloud what it takes to be effective at “squeaky wheel politics.” Seems like no one has solved that puzzle yet.
.

Monday, January 19, 2015

School budget cuts will be (mostly) your legislators’ fault

It looks increasingly like we’re in for more school budget cuts. The Governor has proposed a paltry 1.25% increase in supplemental aid (formerly known as “allowable growth”) for next year. According to our superintendent, that’s well below what it will take to avoid another round of cuts like last year’s. If last year was any indication, the district will probably consider eliminating fifth grade orchestra and band and cutting foreign languages from junior high entirely, and that would only be the beginning.

The blame for this will fall mainly on the Governor and the state legislature. Not only are they failing to provide the needed funding, but they are funding education “reform” ideas like the “teacher leadership” program with money that otherwise could have been made available for supplemental aid. As Karen W. points out, supplemental aid for next year would have been almost 4% (and for the following year, 6%) if that “teacher leadership” money had been used for supplemental aid.

The Johnson County legislators who are complaining about the paltry supplemental aid increase all voted for the teacher leadership program. Did they not realize that it would come at the expense of other educational needs, like music, foreign languages, and smaller class sizes?

With higher supplemental aid, our school board could always have chosen to fund a teacher leadership program by cutting music and foreign languages, if they really thought it was worth it. Instead, our state legislators made that decision for us, without any consideration of the cuts it would cause.

Can’t you just feel your schools getting better?

And if the state Department of Education has its way, the legislature will divert even more funds next year to pay for the expensive new standardized tests that the public is clamoring for. What will be left to cut?

So I have some sympathy for our school board members, who will have to decide where to make the cuts. But I do hope they look closely at how the district is spending its money, and not just at the usual targets. One thing they could do is to take a close look at how much the district is spending on standardized tests, above and beyond those that are required by the state. Here is a chart showing all the standardized tests that our district uses (click to enlarge):


That’s a lot of testing—and so, I assume, a lot of money. Some of it may be required because of strings attached to various revenue sources. Maybe some of it is even so beneficial that it’s worth cutting music, languages, and staff for. Is anyone asking?
.

Because surveillance and authoritarianism model kindness and respect

Just in time for Martin Luther King Day, the Governor has proposed to make our anti-bullying statute even more sweepingly authoritarian. As usual, the bill exhibits zero concern for whether schools might use it in ways that would violate students’ civil liberties.

If there had been a statute prohibiting people from creating a “hostile environment” by discussing someone’s “political belief,” can anyone doubt that King would have been prosecuted under it?

More in the comments here.
.

Sunday, January 18, 2015

Quote for the day

Research documenting the validity of teacher judgment is, at present, shamefully hard to come by. About ten years ago, I read a brief monograph summarizing a few scattered studies that affirmed the validity of teacher judgment. The federally funded, master file on educational research called ERIC lists over 10,000 descriptors available for searching this comprehensive educational research database. As of 2001, “teacher judgment” did not even make list. The extent to which this obvious information asset is overlooked is one of the most appalling phenomena in education today.
George W. Elford, Beyond Standardized Testing: Better
Information for School Accountability, 2002

How could we ever live with less standardized testing? How would we ever know whether the kids were learning anything?
.

How much are we paying for “face validity”?

I learned a new term this week: “face validity.” “Generally, face validity means that the test ‘looks like’ it will work, as opposed to ‘has been shown to work’.” “Some people use the term face validity only to refer to the validity of a test to observers who are not expert in testing methodologies.” Others have equated it with “pandering to stakeholders.”

The idea of “face validity” seems relevant to the state task force’s recommendation that we adopt the very expensive Smarter Balanced tests. The reason the Smarter Balanced tests cost so much more than the tests we’ve been using is that they use computer adaptive technology—varying the questions based on the student’s responses as the test goes along—and include time-consuming “performance tasks,” which purport to “require students to apply their learning to a real-world problem” (in a classroom, on a standardized test).

Not everyone agrees that expensive question types measure “higher-order thinking” and “real-world problem-solving” appreciably better than multiple-choice questions do. The task force’s dissenting member, Karen Woltman, examines some criticisms of performance task assessments here. Iowa City’s H.D. Hoover wrote twenty years ago that “People who think that multiple-choice tests measure trivial facts and performance assessments measure higher-order processes don’t know much about measurement.” I wonder how much has changed in the interim. His talk critiquing performance tasks is a great read.

One thing is true, though: “Performance tasks” and “computer-adaptivity” do sound so twenty-first century! Adding them to our tests enables the state to point to impressive-looking innovations in assessment. How much of the proposed eight-fold (or more) increase in cost is just paying for that kind of “face validity”?
.

Saturday, January 17, 2015

Should ninth-graders be taking AP courses?

I’ve been wishing for years that our district would not teach the kids to chase unreflectively after praise rather than think for themselves about right and wrong. Now I’m starting to think that the district itself does the same thing.

It seems like I can’t go to a City High event or open a City High email without being reminded of the many accolades that City High has been awarded. Many of these are in the form of being named “one of America’s top high schools.” Everyone likes to hear nice things about their kids’ school, but I think most people know to take these assertions with a grain of salt. For one thing, test scores usually play a big role in the criteria, and our district’s high test scores are probably largely a function of its demographics, not its school policies. Are the district’s policies making those scores higher or lower than demographics alone would predict? Don’t ask. (And of course not everyone agrees that test scores are the ultimate measure of the quality of a person’s education.)

A little high school boosterism is to be expected. But if the district pursues certain policies just to chase this kind of accolade, then it can actually do some harm. One of the factors that goes into many of these “nation’s best” assertions is the number of students enrolled in AP courses. The number of AP sections has also been an ongoing bone of contention among people who are concerned about equity issues between City High and West High. These forces have created an incentive for the high schools to enroll as many students in AP courses as they can, regardless of whether those courses are in the best interests of the students enrolled.

I know a number of students who were invited to take AP U.S. History in the first semester of their freshman year in high school. I also know kids who accepted that invitation only to discover that the class was too hard and too much work, and who then dropped it. I’m open to the idea that there might be the rare ninth-grader who is genuinely driven to take a college-level U.S. History course, but that’s not what appears to be happening at our high school. I know at least six kids over the past two years who were invited to take the AP course as a freshman, and I don’t know all that many high school kids.

I have a lot of doubts about the value of AP courses. I would much rather the district craft its own honors-style courses than offer courses that are so single-mindedly focused on passing a standardized test created by some outside entity. Moreover, I don’t believe for a minute that an AP course is a substitute for a college course in the same subject matter. I wonder whether students are actually doing themselves a disservice to take an AP course rather than wait and take the college course (which the AP credit enables them to skip). For a couple of critiques of AP courses, see here and here.

But say what you want about high school juniors and seniors taking AP courses. Freshmen? If the course really is college-level, do freshmen belong in it? If the course is suitable for freshmen, is it really a substitute for college course work? My fear is that neither is true: that the courses are inappropriate for freshmen and overvalued, too.

Does the district’s pursuit of AP enrollments reflect thoughtful inquiry into the value and appropriateness of AP courses? Or it is just the result of chasing whatever the conventional wisdom values?
.

Thursday, January 15, 2015

The money drain

As for which standardized tests the state should require, there’s one other point in favor of the Next Generation Iowa Assessments: not only are they much cheaper, but the money goes to an Iowa enterprise—the Iowa Testing Programs, which is part of the University of Iowa. The Smarter Balanced Consortium is not based in Iowa. Sure, the Smarter Balanced tests are likely to be administered by Pearson, which has lots of Iowa employees, but Pearson is a billion-dollar British multinational corporation. So who knows where the money ends up.

Of course my objections to the role of standardized testing in schools involve much more than cost, but money seems to be the language policy-makers speak, so . . . why the outsourcing?
.

How to contact Iowa state legislators about education policy

The school district here is encouraging people to go to Des Moines on Tuesday, February 17 to talk up the district’s legislative priorities. Contact Chace Ramey (ramey.chace@iowacityschools.org) if you’re interested. (H/t Julie VanDyke.)

The district’s main priority is to convince the state to provide more supplemental state aid, which I’m all in favor of. While you’re up there, though, you might also speak up against the proposal to spend eight times as much (or more) on standardized testing. Low state aid + expensive new tests = more cuts to school programming.

I assume the district’s central administrators generate the list of the district’s legislative priorities. My sense is that school administrators think very differently about standardized testing than the average person does. I wish I knew what our board members thought about spending that much more on the tests.

You can also contact legislators by email. Here are some handy lists that you can cut and paste.

Senate education committee members:

hcqbach@gmail.com, brian.schoenjahn@legis.iowa.gov, amy.sinclair@legis.iowa.gov, jerry.behn@legis.iowa.gov, tod.bowman@legis.iowa.gov, robert.dvorsky@legis.iowa.gov, rita.hart@legis.iowa.gov, rob.hogg@legis.iowa.gov, david.johnson@legis.iowa.gov, kevin.kinney@legis.iowa.gov, tim.kraayenbrink@legis.iowa.gov, liz.mathis@legis.iowa.gov, jason.schultz@legis.iowa.gov, mary.jo.wilhelm@legis.iowa.gov, brad.zaun@legis.iowa.gov

House education committee members:

ron.jorgensen@legis.iowa.gov, tedd.gassman@legis.iowa.gov, patti.ruff@legis.iowa.gov, ako.abdul-samad@legis.iowa.gov, timi.brown-powers@legis.iowa.gov, josh.byrnes@legis.iowa.gov, dennis.cohoon@legis.iowa.gov, cecil.dolecheck@legis.iowa.gov, greg.forristall@legis.iowa.gov, joel.fry@legis.iowa.gov, ruthann.gaines@legis.iowa.gov, curt.hanson@legis.iowa.gov, maryann.hanusa@legis.iowa.gov, jake.highfill@legis.iowa.gov, kevin.koester@legis.iowa.gov, mary.mascher@legis.iowa.gov, norlin.mommsen@legis.iowa.gov, sandy.salmon@legis.iowa.gov, david.sieck@legis.iowa.gov, art.staed@legis.iowa.gov, quentin.stanerson@legis.iowa.gov, sharon.steckman@legis.iowa.gov, cindy.winckler@legis.iowa.gov

Johnson County delegation:

joe.bolkcom@legis.iowa.gov, robert.dvorsky@legis.iowa.gov, david.jacoby@legis.iowa.gov, vicki.lensing@legis.iowa.gov, mary.mascher@legis.iowa.gov, sally.stutsman@legis.iowa.gov, bobby.kaufmann@legis.iowa.gov, kevin.kinney@legis.iowa.gov

Full names of the Education Committee members are here. You can find your own representatives here.
.

Wednesday, January 14, 2015

Marilynne Robinson on standardized testing

Novelist and essayist (and Iowa Citian) Marilynne Robinson, in an interview for The Nation:
I hate the whole business about standardized tests, which implies that everyone should have basically the same aspirations and satisfy the same norms, and what could be more destructive, you know, of the sense of individual personality than that? The thing we know—the thing we know is that people are highly individuated, in terms of their gifts and their proclivities and their interests. We tell ourselves this all the time, but we don’t educate people in a way that makes it possible for them to respond to the fact that this is true. We don’t, you know. I mean it’s more and more regimentation. I think this is a terrible choice, and destructive, and I think that we can only do it because we tell ourselves this thing about our comparative failure as educators.
.

Tuesday, January 13, 2015

What the school board’s not talking about

I hear this post got a mention (during public comment) at our local school board meeting tonight. I’m pretty mystified about why school board members throughout Iowa aren’t kicking and screaming about the proposed enormous increase in spending on standardized tests. When it’s time to make the cuts, those board members will be the first ones to take the heat. Is it just that they’re so used to thinking of themselves as state employees rather than elected representatives that they’re just waiting for the state to tell them what to do? Or maybe they think it’s all a good idea?

Related post here.
.

What the Governor’s not talking about

So the Governor announced some pretty paltry supplemental school aid plans (formerly known as “allowable growth”) this afternoon. No reference to any kind of separate funding of expensive new standardized tests or the technology for them. I suppose that could be a sign that shifting to the Smarter Balanced Assessments is not a high priority for his administration. On the other hand, it could just be a sign that standardized tests, like the Common Core, are unpopular, so the Governor doesn’t want to talk about them. (They will be all the more unpopular if there is no designated funding for them.) I couldn’t help but notice that the state Department of Education initially announced the task force’s recommendation of the new tests in the middle of Election Day and then released the actual report on New Years’ Eve—not exactly trying to get people’s attention. If it’s such a great idea, why aren’t they proudly publicizing it?
.

Standardized tests and your cat’s body mass index

Someone recently showed me this system for measuring a cat’s body mass index. (Don’t ask why.) The system requires you to measure your cat’s girth and lower hindlimb and then plug those measurements into a formula containing numbers that go to the fourth decimal place. Don’t divide by 0.7063; make sure it’s 0.7062.

Four decimal places! Of course, a measurement is only as precise as its least precise input. Call me crazy, but I think I’m a long way from being able to measure my cat’s “lower hindlimb” with enough accuracy to justify using multipliers that go to the fourth decimal place.

I thought of cat BMI when I read the state task force’s report recommending that Iowa adopt the expensive, computer-based Smarter Balanced Assessments:
Computer-adaptive testing, where the computer selects more or less difficult items based on the student’s answers to prior questions, is better able to pinpoint (more reliably and with fewer items than a fixed-form assessment) the performance of students performing at both high and low levels of performance (e.g., students who are gifted, students with disabilities). This means more precise scores for Iowa students.
Better able to pinpoint the performance of students performing at both high and low levels of performance! Now, maybe you’re different from me, and maybe one of your big worries is that the annual standardized tests aren’t pinpointing your child’s performance precisely enough. Even so, do you really believe that any standardized test, no matter how good, could “pinpoint” your child’s academic “performance”?

Schools are constantly reminding us that kids’ test scores can vary depending on how much they’ve slept and what they ate for breakfast. There’s even evidence that scores are affected by random events like variations in air pollution and whether there’s been a recent violent crime near the school. Then there’s the question of whether a student is really trying to do well on the tests, or just trying to get through them—a question that will only grow larger when the tests take twice as many hours. So how precisely can these tests possibly measure your child’s ability?

And never mind that the whole assessment regimen is built on the assumption that the Iowa Core standards are themselves some kind of precision instrument, and that “performance” will be maximized by preventing even small departures from their supreme wisdom. It must be reassuring for education officials to imagine such a well-oiled machine. But there’s no empirical basis for the assumption that the Common Core is precisely the best way to turn your child into a capable adult.

There’s no point in obsessing over precision at the top of the pyramid, when it’s non-precise assumptions from there on down.

The task force would have you believe that standardized tests are like thermometers, and that more money buys you more decimal places in the temperature reading. If you believe that, I’ve got a Feline Fitbit I’d like to sell you.
.

Friday, January 9, 2015

No price too high for standardized tests?


[This post appears as a guest opinion in the Des Moines Register and Iowa City Press-Citizen; I’m posting the published version here with some additional notes afterward.]

Should we spend three times as much on standardized testing as we currently do, or eight times as much?

That question captures the range of current establishment opinion. The establishment answer, of course, is eight times as much—and more.

Currently, Iowa requires schools to administer the Iowa Assessments every year. Because those tests are developed by the University of Iowa, we get them at cost: about $4.25 per student.

But in 2013, the state created a task force to determine whether to require a different set of assessments as of 2016-17. The task force considered two possibilities: the Next Generation Iowa Assessments and the Smarter Balanced Assessments. The former would cost $15 per student. The latter cost significantly more and would need to be supplemented with a separate science test, for a likely total cost between $30 and $40 per student. Those tests would also require unknown millions to buy and maintain the necessary technology; the task force—incredibly—didn’t even try to estimate the cost.

The tests we’ve used for years are now inadequate, we’re told. We need tests that incorporate “constructed responses” that require human graders, or tests that the students will take on computers, with technology that can adjust the difficulty of the questions to the student’s responses. We need to make the kids sit through twice as many hours of testing as they do now.

The task force recommended the most expensive tests (though not unanimously). If you suspect there are diminishing returns from all this expensive additional testing, you’re a Luddite.

What will get cut to pay for these tests? No one will say. Don’t believe for a minute that the state will fund these new tests at no sacrifice to other educational funding. Every dollar the state spends on testing will be one less dollar available for general school funding—at a time when many districts are already making severe budget cuts. No matter how it’s spun, the school districts are about to get hit with an enormous unfunded mandate.

Standardized testing is now the Defense Department of the school budget: only the most deluxe, big-ticket, exorbitant program will do. Never say no, regardless of what has to be sacrificed. The testing companies get richer; your kids’ education gets poorer.

The only thing standing between us and this enormous increase in spending on standardized testing is our legislature. If your school district has to make class sizes larger, cut band and orchestra, eliminate foreign language classes, or worse—all so we can have the shiniest new standardized tests—you’ll know who’s responsible: state legislators who decided that no price was too high.

________________


Here’s some additional information that I couldn’t fit in the guest opinion:

By not even trying to quantify the costs of the technology that will be necessary, the task force violated the legislature’s explicit charge. The act creating the task force provided that “the task force shall consider the costs to school districts and the state in providing and administering such an assessment and the technical support necessary to implement the assessment.”

The dissent to the report focuses largely on the fact that the task force did not assess the full costs of the Smarter Balanced tests. You can read it in full here.

________________


The task force report works hard to minimize the actual cost of the Smarter Balanced Assessments, stating that the tests will costs about $22.50 per student. In fact, the cost will be significantly higher, for several reasons:

First, the $22.50 figure is an estimate that comes from the Smarter Balanced Assessment Consortium—that is, by the people who have a major interest in getting the state to adopt the tests. In fact, as the Consortium admits, the actual cost will depend on what is charged by the private vendors (Pearson?) that will administer and score the tests. If those charges come in higher than the Consortium’s estimate, no one will be surprised.

Second, the Consortium’s additional “interim” tests add another $4.80 per student to the cost. The report refers to those tests as “optional,” but its inclusion of this chart seems to promote their use. (These interim tests would also add to the number of hours the kids have to sit for tests.)

Third, Iowa law requires a science assessment. Unlike the (far less expensive) Next Generation Iowa Assessments, the Smarter Balanced tests do not include a science assessment, so the state will have to pay for a separate science test. That test could cost another $10 per student.

Fourth, if some districts are not ready to conduct the tests via computer, they can use paper-and-pencil versions of the tests—for $10-12 more per student than the online versions.

For those reasons, I used an estimate of $30 to $40 per student, and that’s just for the tests themselves, not for the necessary technology. I would have preferred not to estimate at all, but the task force didn’t assess those costs itself.

________________


Then there are the technology costs. Even the task force report admits that its estimate of the per student cost does not include the tech costs that will be necessary in districts across the state of Iowa—which, as the dissent points out, are likely to be “significant and ongoing.” What will those costs be? Don’t ask the task force.

The tech costs won’t just be for equipment and bandwidth, either. They will almost certainly require ongoing expenses for tech staff to maintain the tech infrastructure, as well as ongoing upgrades. There goes a teacher.

On the lack of information about the tech-readiness of Iowa’s school districts—and on some of the debacles experienced in other states—read Karen Woltman’s post here.

________________


The task force report argues that using the Smarter Balanced tests could enable districts to save money by eliminating some other tests they are currently using. But if that’s true, it’s also true of the much less expensive Next Generation Iowa Assessments, and the task force doesn’t claim otherwise. Moreover, this argument conveniently ignores the fact that the Smarter Balanced tests do not include the statutorily-required science assessment—which means the Smarter Balanced tests will actually cause expenditures on additional tests.

________________


Though it works hard to minimize the cost of the new tests, the report also works hard to make the current tests look more expensive. Although the Iowa Testing Program lists the cost of the current tests as $3.50 per student, the report says that they cost “$4.25 to $6.25” per student, citing a document that is not available online.

The $4.25 figure supposedly comes from adding a $.50 processing fee and a $.25 barcode fee. I’m willing to trust the report about the existence of those fees, though a source would be nice.

The $6.25 figure is what districts would pay if they didn’t agree to allow some field-testing of new questions—which means no one has to pay it.

Then the report asserts that the current Iowa Assessments have a hidden $2.25 additional per-student cost because of “additional data managing and reporting” costs borne by the state. I didn’t include that figure in my estimate, for three reasons. First, the report gives no citation at all for that assertion. Second, there is no way to tell whether the report is comparing apples with apples: will there be similar “data managing and reporting” costs if we use the Smarter Balanced tests? Third, if the report is just counting money the state spends using the test results, it can’t fairly include that as part of the price of the tests.

Moreover, even if you include the $2.25, you’re only up to $6.50—a far cry from what the Smarter Balanced tests cost.

________________


One of the most striking paragraphs in the entire task force report is worth reprinting here in full:
The other costs that cause concern are related to districts’ technology readiness to support online testing. While there is no doubt that some districts are behind in technology readiness, schools will not be required to make devices and internet connections available to each and every child simultaneously. As the Consortium notes, “A 600-student middle school could test its students using only one 30-computer lab.” In these ways, the costs of upgrading school technology infrastructures are not likely to be overly burdensome on the whole. Besides, these are costs the state of Iowa should shoulder. We must better incorporate technology into the delivery and conduct of not just our assessments, but our instruction as well. The future of assessments is online, but so is the future of teaching and learning. Investing in devices and bandwidth is necessary and should be done by the state regardless.
Well, the Consortium assures us that we can test 600 students with just 30 computers, so we don’t have to bother quantifying technology costs. You might think giving 600 seven-hour-long tests on just 30 computers would be “overly burdensome,” or, say, an unimaginable nightmare. But the salesperson lobbyist Consortium told us it’s okay, so it’s cool.

But anyway, we should buy a lot of technology! We’re sure it will be useful and necessary, for lots of stuff that we can’t actually identify because we’re just an assessment task force. But that’s what the future’s all about, right—computers? So there’s no need to consider how much it will cost, even though the legislature told us to.

Okay?

________________


Don’t even get me started on the assertion that “the future of teaching and learning” is online.

________________


Notice the double standard. When the report wants to make the existing tests look expensive, it includes extra costs that it claims are borne by the state. But when it wants to make the Smarter Balanced tests look less expensive, it argues that “Besides, these are costs the state of Iowa should shoulder”—as if that somehow means they don’t cost the districts anything. Of course, any money the state spends on testing is that much less money it can spend on other educational needs, including supplemental aid to the districts.

________________


Although the task force didn’t quantify the total cost of adopting the new tests, it assures us that this unknown cost is “proportionately small,” compared to the total education budget. So what? It’s also small compared to the national debt, or to the gross national product of Bulgaria. What matters isn’t how much of the total budget it is; what matters is how we’re going to pay for the increase. What will get cut to pay for these tests? If we don’t know, how can we know whether the tests are worth having?

Instead of giving us a cost-benefit analysis, the task force gave us a benefit analysis—really, closer to an advertisement. Is that what the legislature asked it to do?

________________


One reason the Smarter Balanced tests are so expensive is that they use “computer-adaptive testing,” which means that the technology will vary the difficulty of the questions in response to a student’s answers as the test goes along. There is not universal agreement that computer-adaptive testing is even desirable, though, let alone worth paying for. Commenters make some interesting points about it here.

________________


The report emphasizes that computer-based tests will enable districts to get the test scores back much more quickly. Is that worth paying a lot of money for? The tests are going to be given toward the end of the school year. If Susie gets a lower-than-hoped-for “problem solving” score, will the school somehow re-teach that year’s math curriculum to her in the last month of school? How would they even know what to fix?

If there is a connection between the quick score reporting and the improvement of your child’s education, you won’t find it explained in the task force report. Am I wrong to think that, at best, the scores will be used primarily to evaluate curricular choices and recommended teaching strategies, which schools don’t change overnight? (Large bureaucracies aren’t exactly known for being nimble.) If the ultimate point of the scores is to improve the curriculum next year and beyond, what is gained by getting the scores in days instead of weeks?

________________


The report also asserts that the new tests will provide “more precise scores for Iowa students.” I have a lot of concerns about K-12 education, but I have to admit that I have never complained about my kids’ scores on the annual standardized tests not being precise enough. In any event, call me skeptical that any test will ever “precisely” measure a kid’s “academic performance.” (See this post.)

The report also asserts that the tests will measure kids’ “ability to transfer their learning to real-world situations”—by giving them a standardized test in a classroom. Again, call me a skeptic. (See this post.)

The report also asserts that the tests will help our students “compete against students across the country and beyond” in “today’s global economy,” because the same tests have been adopted by “21 states and a U.S. Territory.” Now Iowa can finally prevail in its age-old rivalry with the U.S. Virgin Islands.

On those topics, more in upcoming posts. In the meantime, I can’t resist raising one question: how much do you want to bet that the same kids who do well on the current (inexpensive) tests will also do well on the new (very expensive) ones? If so, then what is actually being measured?

________________


What’s missing from the report is any concrete explanation of how these tests will be used to improve your child’s education. There are all kinds of fun, expensive things you can do with data. Many of them have no meaningful bearing on real life. A marginally more “precise” “total writing” score might be a victory for the test designers, but just how does it make your child’s life better? Through what series of events does that occur?

I came away from the report with the discouraging sense that our state education bureaucracy has really lost its way. Nobody wants to admit that the ability of empirical data to tell us how to turn kids into capable, independent adults is severely limited. Standardized testing has legitimate uses, but it’s not vested with magical powers. It’s almost as if education officials have constructed a reassuring, imaginary universe in which their job is to manipulate abstractions—like in those infernal box-folding questions on the tests—rather than to deal with the messiness of the living, breathing, infinitely varied bodies and minds who arrive in the classrooms every morning.

They can dream. But if they want us to spend millions of dollars, they owe us much more specific, realistic information about costs, benefits, and what will get cut.
.

Monday, January 5, 2015

When “consider the costs” means “don’t consider the costs”

When our legislature created the state assessment task force, it explicitly required that “the task force shall consider the costs to school districts and the state in providing and administering such an assessment and the technical support necessary to implement the assessment.” Yet the task force’s report, which recommends adopting very expensive new standardized tests, did not even try to estimate the costs of the technology and tech support that will be necessary to make those tests work.

Karen Woltman, the only person on the task force simply by virtue of being a parent, dissented from the recommendation because of its failure to consider the full cost of the new tests. Here is her dissent in full:


The Smarter Balanced Assessments are by far the costlier of the two assessment options in front of the Task Force. Whether the Smarter Balanced Assessments are worth the additional costs cannot be determined without quantifying all of the costs involved. This has not yet been done.

The information reviewed by the Task Force shows that the Smarter Balanced Assessments will take more than twice the amount of time to administer as the equivalent portion of the Next Generation Iowa Assessments and do not include a required science assessment. Science and social studies assessments can be added to the Next Generation Iowa Assessments for a total test administration time that is still 2 to 3.5 hours shorter than the Smarter Balanced Assessments alone.

The information reviewed by the Task Force shows that the Smarter Balanced Assessments will cost more per student, at an estimated $22.50 for the summative assessment only, and that those costs do not include a required science assessment. The Next Generation Iowa Assessments can include a science assessment for an estimated total cost to Iowa schools of $15 per student.

However, the Task Force lacks adequate information about the costs for school districts and the state to build and maintain the necessary school technology infrastructure to administer the Smarter Balanced Assessments. No comprehensive survey of the current state of school technology infrastructure has been conducted yet; consequently, these costs have not been quantified and are unknown at this time. The limited evidence in front of the Task Force suggests that these costs will be significant and ongoing. Even if the Legislature were to appropriate money for these costs, the appropriation would likely come at the cost of reduced supplemental state aid and thus would be in effect an unfunded mandate.

At the outset of our work, task force members agreed that our recommendations should be guided by what is best for Iowa’s children. Accountability testing is something we do for the adults, great instructional programming–including high quality art, music, world languages, and extra-curricular programs–is what we do for the children. Ultimately, it is best for Iowa’s children to obtain the accountability data required with the least impact on instructional programming possible. The Smarter Balanced Assessments divert more time and money from instruction than necessary for accountability purposes, and for these reasons, I respectfully dissent from the task force’s recommendation to adopt the Smarter Balanced Assessments. Based on the evidence currently in front of the Task Force, I would recommend adoption of the Next Generation Iowa Assessments instead.

Read Karen’s posts about her experience on the task force here.
.

Sunday, January 4, 2015

The series of posts that dares not speak its name

Reading through the report of the state’s assessment task force—which recommends that the state adopt very expensive new standardized tests, without even quantifying the total cost—is enough to bring the dreaded word blogathon to mind. I’m not crazy enough to commit to another blogathon. But let’s just say that there is an awful lot of raw material for blogging in that report. My free time is very scarce lately, so it would be unwise to promise so many posts in so many days, but some of these posts will practically write themselves. In other words, stay tuned.
.