I just engaged in a lengthy (and ongoing) Twitter exchange with the director of our state’s Department of Education, Jason Glass. Earlier this week, the Department held up as exemplary a school district that cut recess time to raise literacy scores. (Teachers and students “weren’t happy with some of the things we had to drop, such as morning recess time because we really don’t need that,” one principal said.) Many people chimed in to point out that there’s no reason to think that cuts in recess help kids learn. What I wanted to know was how the department could be sure that the test score increases represented meaningful education, as opposed to just test prep. If there is a difference between the two, and Glass agreed that there is, then the test scores themselves can’t help you distinguish one from the other. So I asked what Glass’s criteria are.
You can be the judge of whether the discussion is going anywhere. But it seems clear that Glass must have a different definition of test prep than I have. Mine would be: single-mindedly pursuing higher test scores at the expense of other values that are cumulatively more important. One such value is providing a humane school experience. Another is preserving a child’s enjoyment of learning. Another is not teaching authoritarian values. Another is giving enough attention to subjects that aren’t tested or don’t lend themselves to testing.
I suspect his definition would focus more on whether the test score increases reflected a meaningful improvement in the particular quality they purport to measure. That would be a defensible definition, but it doesn’t excuse him from asking what has been sacrificed for the sake of that improvement. How does the DOE measure those sacrifices? I don’t think it does. If it did, how would it determine what sacrifices are “worth it”? Though Glass keeps referring to evidence-based practices, the latter question cannot be answered empirically, because it entails value judgments.
How does the DOE make those judgments? In what meaningful sense are they informed by the values of the families they affect?
.
11 comments:
Read Scott McLeod's excellent post scrutinizing the supposed "success" of the district that cut recess.
Even under the narrower definition of "test prep," I don't see how Glass has ruled out that those scores were the product of test prep. Glass suggests he can do so by a combination of test scores and teacher/administrator judgments. But the scores can shed no light on whether they themselves are the product of test prep, and educators are unlikely to confess to engaging in test prep to raise scores.
Thanks for the dialogue Chris.
As I've made clear in our exchange several times, there simply is no evidence to support your allegation of "test prep" being the reason for increased results.
I'd characterize test prep as simple fact/recall, memorization of likely test facts, guessing strategies, and test time management strategies.
Yet in the schools we profiled, we see none of those strategies being employed. We see teacher instruction individualized to each student, improved curriculum alignment with high quality standards, added reading support and assistance to struggling students, student developed goals, an evidence-based reading strategy, dedicated time for reading daily, and integration of reading into other areas of study.
This is what the educators in these schools told us they were doing. None of it is "test prep." Yet, you maintain the improved results must be somehow achieved by gaming the assessment.
Perhaps this is some grand conspiracy where "test prep" was used to achieve success in these schools. Yet, there is no evidence of this ... only your persistent and unfounded allegation.
Or perhaps, the dedicated educators and the hard-working students in the schools did the work that actually resulted in better readers.
Wouldn't that be something?
JG
Jason -- I don't know how many times I can say that I am not alleging that those schools engaged in test prep. I am alleging that you made no effort to determine whether those increases were the result of test prep. The test scores can't help you determine that, and the school is obviously not going to self-disclose that they are using questionable methods.
But fine, you have a much narrower definition of test prep than I have. Why won't you answer the larger question: What is the point of measuring the benefits of a given educational program, if you don't also measure what's being sacrificed?
What good are high "literacy scores" if the child does not become literate. They can call words all day, pick out theme, main characters, plot, etc. by rote. But if they don't learn to enjoy reading and if they don't consider reading an integral part of their lives, or if they don't have experiences, including recess that they can relate to what they are reading, are they truly literate?
This Jason person, superintendent I assumes to come to the conclusion that the results of learning is only tangentially related to the almighty test score. What is learned for the test,is usually forgotten in days. What is learned by experience and what relates to life is what stays with the child.
I believe Chris is concerned about the narrow focus of evaluating successful teaching/learning as purely the result of academic assessments. Our goal in education - in order to prepare students to enter our world as competent, thinking, contributing adults - is to educate the whole child. While I understand that funding issues that are tied so narrowly to quantifiable results necessitate high stakes testing, I share Chris' concern that the emphasis on test scores as "definers of success" risk neglecting the preparation and education of the whole child.
The ASCD has an interesting initiative, "The Whole Child Initiative" http://www.ascd.org/whole-child.aspx that goes deeper into this whole-child approach to education. Included is a free School Improvement Tool designed to evaulate a school's or district's approach to whole child education.
Chris, this may be the tool you are looking for to evaluate the "expense cost" of cut activities and programs in your schools.
Goes against everything I've ever read about brain development research.
Kids need to move frequently.
NIH has solid evidence that recess is necessary and it improves academic outcomes:
http://newsinhealth.nih.gov/issue/jun2012/feature1
@Jason Glass - with all due respect, do you think the teachers in the schools you "profiled" would admit to test-prep? You must not have much experience in the classroom? Or maybe not since NCLB or RttT?
I have seen some things, trust me. And I would not admit to them if my school was "profiled". If I really wanted to, during my multiple decades of experience, I could have gotten teachers fired for what I saw. This is widespread stuff - stuff only people on the ground know about. Tricks of the trade, if you will.
We spend millions every year on test prep materials (for SAT and others), and those materials have been shown to garner some score improvements. But what I've seen is worse - teachers so knowledgeable about what's actually on the test, especially after years of proctoring, administering, performing read-alouds, etc..., that I am suspicious of the results of any state imposed standardized test scores.
In fact studies have shown that after 5 years of administering the same test from state departments, scores GO UP.
And principals have been guilty of narrowing the curriculum since NCLB. In a middle school I once taught at, science and social studies were sacrificed in order for test prep time on math and reading. Principals have been pulling this crap off for years. Principals at my high school hire outside tutors for classes that are used to calculate AYP, yet now that we ALL are tested (we are a RttT state) this sticks out like a sore thumb. My growth scores using EVAAS are calculated from a class that the principals hire outside tutors for test prep. This means it will be harder for my students to show growth as their growth expectation is calculated primarily from this AYP course that the students receive test prep for, which in turn inflates their score and increases their growth expectations for my class.
I trust NO high stakes tests from state departments. When teachers teach to the test, especially when they know what's on the test, this is grade inflation and it invalidates the test. In fact, test prep by itself invalidates the test. Teaching is not supposed to be dictated by what's on a test - testing is a secondary part of education where no stakes should be attached and school personnel meet behind closed doors to discuss in order to make adjustments.
The way I see it, if archaic and inauthentic tests are going to be given, there are going to be teachers who engage in test prep, or, as the teachers in my district in Alberta call it, "teaching to the test." In my district teachers proudly admit such practices citing they have no choice. If all we taught was what students needed to succeed in the 21st Century, students would do poorly on these standardized tests. I refuse to cater to the tests, instead choosing to prep my kids for life. Chis, I'd say the larger question is why are we still requiring tests that measure skills that no longer matter? I teach ELA and I shudder at the thought of asking kids to determine the "correct answer" from a text they read. I'd rather my students think critically for themselves and be able to defend their answers. This is a skill I see as useful. I blogged about this bigger issue at http://jondavidgroff.com/2013/03/15/teaching-for-21st-century-but-assessing-in-the-past/
Post a Comment