Yesterday I came across a checklist that our district uses in its sixth-grade curriculum. The students are learning “informational writing,” and are supposed to make their essays conform to the rules that appear on the checklist. Those rules include:
Does the introduction also introduce the subtopics?The checklist proceeds through thirteen such rules. Then, without any recognition of irony, the checklist concludes:
Are there at least three subtopics, with each subtopic written as a separate paragraph?
Does the conclusion mention the main idea and the subtopics?
Are there overused words? Replace overused words with synonyms.
Look at the start of all sentences and paragraphs. No two sentences in the same paragraph should start with the same word. No paragraphs should start with the same word.
Is there voice in the writing? Does the writing sound like you, or could it be written by anyone?As a writing teacher myself (at a law school), I know that it’s sometimes helpful to make generalizations, to describe prevailing conventions, and to give students rough templates for different forms of writing. But even generalizations should have good reasons behind them. Several of the rules in this checklist are ridiculous, and are not true of any conceivable form. Others might apply in some circumstances, but certainly not to all “informational writing.”
What possible explanation is there for this checklist other than to satisfy the expectations of some standardized writing assessment? (Let me guess: Evidence shows that the checklist increases student achievement – as measured by standardized writing assessments!) I don’t know which would be worse: that it’s designed to teach to a test, or that someone actually thinks it’s good advice.
The checklist as a whole sends the message that expression is independent of content, and that good writing is a matter of applying formulas. It’s not; it’s a matter of exercising judgment. So is good teaching. But as education policy has become obsessed with quantitative measurement, the concept of judgment has fallen on hard times – even in Iowa City.
(Click to enlarge)
.
20 comments:
In Measuring and Managing Performance in Organizations [1] Robert D. Austin argues something like: All (or essentially all) high stakes measurement are going to lead to sub-optimal behavior and has a high chance of leading to pathological beharior. The essence of the argument is that what one wants people to do is a mixture of the easy to measure and the nearly or entirely implossible to measure. So when a measurement system is put in place only some of the desired behaviors or results are measured. The ones where it is possible to measure. Initially one sees an increase in desired results, because people will increase the measured output. But over time, people do what is measured and rewarded and they will decrease the desired but unmeasured activities to be able to devote more time and energy to increasing the measured ones. Given time and high enough stakes people will engage in totally unwanted behavior, in other words people will start cheating. See any number of CEOs who cooked the books to get bigger performance bonuses or rake in more money on their stock options. And now the cheating by teachers and staff coming to light at various schools.
Mr. Austin also argues that measurement systems can backfire even without the high stakes and covers about some other short commings in measurement systems, in general.
Note, I belive I have the general gist right, but I have not read this book in quite awhile.
I was suprised to see that the book has nothing but five star reviews on Amazon. [1] I've talked to people who really don't like this book. "He points out the problems, but doesn't give any alternatives. I can't change away from measurement and performance pay without an alternative." Well he does give alternatives but some people don't like them and dismiss them out of hand as not being real alternatives. For example Mr. Austin compares American use of variable pay Japan's pay which tends to be seniority based. And without the lavish pay for "hero" executives. based on measured performance. People don't like his alternatives.
I used to think NCLB was flawed based on my reading of Mr. Austin's book and other sources on problems in performance management. But I thought that public education in the US was bad enough that NCLB couldn't public schools any worse and just might improve. (Talk about rose colored glasses.) I was deeply wrong about that. It has done a lot of damage. I still believe that the intentions of at least some backers of NCLB were honorable. That they wanted to improve education for students in public schools. And especially the part of the law that required a breakout of subgroups performance so that schools could not hide behind a good average performance from the majority white population of the school.
I think Mr. Austin's book presents a very good argument for doing away with, and stopping the dysfunction created by, high stakes testing. It gives a highly probable model and explanation for why the dysfunction will happen, and is not just bad apples at particular schools doing the cheating. And I would recommend it to anyone who wants an understanding of why high stakes testing is not working and will not work to create the sort of educational experience we want for our children. Unless there is someone who really only wants their child to learn how to score well on certain tests. I found a nice review that does a more thorough job summarizing Mr. Austin's book. [3]
[1] http://www.dorsethouse.com/books/mmpo.html
[2] http://www.amazon.com/dp/0932633366/
[3] http://bookreviews.dynamicdataworks.com/archive/2008/04/21/Measuring-and-Managing-Performance-in-Organizations--Robert-Austin.aspx
Chris, it does seem heavy on style over substance. With style meaning more "easy to score with a quick glance and a checklist" and less "effective at addressing the chosen topic."
I suppose this is similar to the five-paragraph essay writing that my teachers taught in elementary school but I have never heard the "do not start paragraphs/sentences with the same word" rule before. Also, I think my teachers knew the five-paragraph essay practice was formulaic; I guess they assumed we'd find our voices later.
I'd agree--the last question is incredibly ironic.
I just posted a paper assignment (for the genre of analytic literary criticism) not long ago in which I went out of my way to tell my students it was OK by me for them to use "I." I'd rather they use "I" than offer up tortured constructions such as: "In this paper it will be argued that . . . ." I think for my next assignment I might go further and tell them it's OK by me for them to use the same word at the beginning of two consecutive sentences. I might even go further than that and tell them it's OK by me for them to use the same word at the beginning of every sentence in their entire essay.
I am enjoying this beautiful weather we are having here in Iowa City. I hope you are enjoying this beautiful weather too, Chris. I enjoy reading your blog but enjoy even more taking walks outside near your house in this beautiful weather. I have to go now.
Sorry, Chris--just couldn't resist poking some fun earlier at the check sheet.
Like you, I do understand that part of what it means to teach writing is that you introduce various conventions. But a convention is not the same thing as a rule. Do you know anything about how this check sheet was presented in class? Were students given rationales for the various dictates?
It would also be interesting to know how long this check sheet has been used, whether the same check sheet is given to all 6th grade students regardless of their teacher, and so on. The answers to those sorts of questions would probably shed some light on the issue of the extent to which the district is indeed "teaching to the test."
Shannon--thanks for the reading tip! This book sounds intriguing to me.
Shannon -- Thanks for commenting! That's a great description of why high-stakes assessments have unintended harmful consequences. (Also see this post.)
I wonder what our state's assessment guru (er, I mean, Education Director), Jason Glass, would have to say about Austin's point.
Thanks, Karen and Doris. This checklist does seem to be geared toward generating the notorious "five-paragraph essay," which we are all called upon to write so often in our adult lives. I'm sure the teachers do know that it's formulaic, and at some level at least some of the kids must know it too. I'm not sure that's much of a defense of the enterprise.
I agree that a good teacher could take the edge off of some of this. But this isn't a case of "Learn the rules first, then later you can depart from them." Yes, a teacher might say, "It's not that you should never use first-person, but there are some types of writing in which you normally wouldn't be referring to yourself." But as to some of these rules, the only way to take the edge off would be to say that the rule makes no sense at all, ever. Why put the teachers in the position of having to explain to the kids that what the checklist says isn't literally true, and is in some cases outright false?
(Another nice little irony, by the way: "No two sentences in the same paragraph should start with the same word. No paragraphs should start with the same word.")
As for the weather, it sure is beautiful. I've been parked on the front patio with my feet up as much as possible -- with only my iPhone to distract me. It's funny -- when your comment came through, Doris, all I had to do was glance up to see if you were walking by at that moment . . .
What standardized writing assessment is that checklist preparing students for? High stakes testing requirements for NCLB and in the state of Iowa do not include writing. You can't be "teaching to the test" if there is no test!
The checklist is plainly awful. But is seems to me that beginning writers will need some sort of formula to move from sentence writing to paragraph writing to essay or research paper writing until they have enough practice under their belts to have a feel for organizing different styles of writing.
Imagine telling an elementary student to go write a ten page research paper without any hints about outlining or how to paraphrase or how to properly quote sources or how papers of that length might be organized.
I wonder, now that I have thought about it some more, how much of that checklist is about test prep, and how much of it is about trying to change writing habits of kids who may have been asked to journal their feelings for years, and how much of it reflects how elementary teachers have been trained to teach writing?
How much of it is about high-stakes testing and how much of it is about educational philosophy?
This is just another painful example of a focus on blind skills instead of habits of mind, critical thinking capability, and flexibility of thinking.
Using that checklist makes writing simple for students. Ironically, the checklists are written by the same people preaching "rigor." When will they realize students play the game that's set up for them? What a shame it is that we teach writing as a function of filling in a checklist. Reminders? Sure. Expectations? You bet. But please, do those things through conferences with students who are writing for real reasons to real audiences. Ask kids "why?" a lot. "Why did you start the paragraph that way? Here, take a look at some mentor pieces and compare - what do you notice?"
But that's difficult. Checklists are easy.
And the biggest irony you uncover is this: ask ANY teacher whether or not they think the formulaic writing of standardized tests is valid, and they'll say "no." Then they'll proceed to dole out formulas for their standardized assignments. Hmm.
What experiences would a student need in order to put his/her voice into a piece of writing? That's the question we should begin with, not "What does a student need to do in order to amass enough points to satiate their need for a high GPA?"
Jen -- Thanks for that info. Again, if the district is teaching this stuff because they think it's a good idea, I'm not sure whether I should feel better or worse.
Karen -- On the one hand, I'm not alone in thinking that this kind of teaching is increasingly common because of the use of standardized writing assessments. FairTest, for example, in describing how high stakes testing leads people to teach to the test, uses the five-paragraph essay as one of the prime examples: "Instruction starts to look like the tests. For example, reading is reduced to short passages followed by multiple-choice questions, a kind of 'reading' that does not exist in the real world. Writing becomes the 'five-paragraph essay' that is useless except on standardized tests."
On the other hand, I do think there is a certain amount of actual enthusiasm among some educators for the "five-paragraph essay" as a teaching tool. I don't buy it, but I could imagine it being done in a way that didn't lead the students to think that there are rules to writing an essay. (A couple of interesting links on it here and here.) You might be onto something about how it could be driven by undoing some of the habits developed by other types of writing the kids have been asked to do.
I particularly agree with this passage (from that first link):
"How do I create writing assignments that encourage risk-taking and mental growth without letting good organizational strategies go by the wayside? The answer is not, of course, to turn to alternative methods of organization that presume to fit every writing situation in the academy. These methods have just as much potential to become 'lock-step' as does the five paragraph theme. Rather, the answer is to revisit the pedagogical theory with which I first embarked, starry-eyed, on teaching: that every writing assignment poses a unique rhetorical problem." The writer then describes how the goal should be to help the students figure out how the content, audience, purpose, etc., of each assignment drive decisions about organization and expression.
Which may just be to say, teaching people to write well is hard. But teaching them something else that happens to be easier seems unlikely to lead to any real benefits, and is more likely to mislead students than benefit them. Exhibit A is this checklist.
For what it's worth, my daughter tells me that they do have writing assessments twice a year. Whether they are required by Iowa or NCLB, though, is another matter.
Vini – Thanks for commenting! I totally agree. Teaching writing with a checklist like this is a lot like teaching “good behavior” by simply telling students what to do (a la PBIS) instead of helping them think about how to make choices. They both circumvent the development of reasoning and judgment for the sake of getting the kids to comply with some short-term, more easily assessable goal.
The ICCSD Comprehensive School Improvement Plan shows the district administering the Iowa Writing Assessment in grades 4, 8, and 11.
http://www.edline.net/files/_VMCld_/428ab412aa262b143745a49013852ec4/2008-2013_csip_report.pdf
http://itp.education.uiowa.edu/other_pgms/writing.aspx
Is it possible the writing curriculum is aimed at prepping the students for these exams?
Chris -
I'm an administrator and curricular supervisor in a K-8 school. I'm terrified of the day they discover PBIS because there will be a major push for it. Here, I'm meeting intense resistance for speaking to kids with respect, discussing problems with them, and setting up long term solutions. It seems my job should really be to "strike fear" and make them cry. You might chuckle at that, but I've been told to do more of both of those things!
The point is you're exactly right: school is a behaviorist's utopia. Do kids know how to walk down a hall? Not according to school procedure. Do kids know how to sit and persist in a meaningful task? Depends on who determines the meaning!
So it filters into the curriculum, because why would kids actually DO the curriculum if, A. it wasn't a means to an extrinsic end (grades), and B. it wasn't prescribed specifically? The simple answer is that except for the kids who are especially trained, they won't do those assignments! By middle school, they don't, and you have kids with excellent argumentation skills, an ability to see the big picture, and an uncanny skill for algebra (balancing life and school by NOT doing their busy-work homework), failing.
So is school for learning?
Soon my state will adopt more checklists, this time for teachers. Administrators will lose the ability to model the proper learner/mentor relationships, except for some savvy rebels.
Thanks for your blog and your "radical" (as in, root) commentary!
Karen -- Thanks for that info. Here is the clickable link to the description of the Iowa Writing Assessment. Unfortunately, your first link is to our district's impenetrable website, and thus takes you nowhere. If you email me the PDF, I'll put into linkable form. At some point the district may have to start paying us a salary for making these documents publicly available.
Vini -- Strike fear and make them cry -- unbelievable! (But, alas, believable.)
Generate meaningless, alienating tasks, and then generate behavioral techniques to get kids (and the teachers) to do them: I can't see how it's good for kids, but I can see how it would create a lot of work for educational bureaucrats.
The details of this checklist aside, I definitely believe that learning how to write in one specific form and context can help students understand how to work in other forms later on. But, as Vini pointed out above, the value would be in discussing why, given the form and the subject matter, it makes sense to use one strategy over another. And, as Shannon pointed out, the temptation to skip that discussion would be strong if the kids (and therefore the teachers) are assessed only on how well they comply with this particular form.
Moreover, the five-paragraph essay is such an artificial creature that it’s hard to see how you could reason from its intended audience and purpose to its typical features. So it’s hard to see how you would be able to extract generalizable principles from that exercise that would be of value in later, real writing tasks that one might encounter.
Instead, it seems like a kind of giving up on the idea that most kids can be taught how to write well. “If we shoot for that, we’ll fail, so let’s shoot for something much lower, and then we can succeed.” Yet this approach is made all the more likely by the use of high-stakes testing – brought to you by the people who accuse their detractors of “the soft bigotry of low expectations.”
Thanks to Karen, here is the district's Comprehensive School Improvement Plan.
I am a student teacher, and during my last placement my cooperating teacher was teaching the students how to write an essay using the "five-paragraph essay" formula. Some of the students objected to the formula, saying that real writing doesn't follow a formula. He didn't appreciate being contradicted and told them that they were wrong. I really wanted to tell him that, actually, the students were correct. But (am I allowed to start a sentence with "but"?) I knew it would be a waste of breath to try to convince him otherwise. I truly think that high school teachers do their students a disservice when they oder students to write using a strict formula. Suggesting guidelines that will help students organize their thoughts and ideas is acceptable, but to make students adhere to a checklist/formula is asinine!
It's discouraging to see such a disconnect between what is being taught to students in high school and what is expected of them in university/college.
Post a Comment