Wednesday, December 16, 2009

Intentions and results in school reform

Everyone is into school reform these days, and everyone has good intentions. Schools and teachers talk about how they're trying different things; every educational symposium features sessions about their innovations, with sympathetic audiences crediting the speakers for trying to "do something."

And it turns out, according to a very big new study, written about here in Education Week, that all this freelance innovation may not be as effective as following very specific scripts.

Thirteen years ago, researchers from the Consortium for Policy Research in Education set out to study three packaged school reform curricula: Success for All, America's Choice and Accelerated Schools. Success for All features scripted lessons which tell teachers exactly how to teach a concept. Accelerated Schools allows teachers to devise lessons, with guidance toward outcomes. America's Choice is somewhere in the middle. After studying the achievement of thousands of students at hundreds of schools, the researchers found that Success for All outranked the others. While the Accelerated School teachers felt they had far more autonomy in their classroom, this autonomy translated into not being very effective, or at least doing no better than the control schools. The Success for All scripts, on the other hand, raised achievement levels of the average participating student from the 40th to the 50th percentile over 2.5 years.

This is a fascinating result and, if you are a teacher, a wee bit tough to swallow. We think of teaching as a profession, benefiting from a personal, charismatic touch. It's somewhat akin to how we think of medicine. And there's a parallel to the dust-up going on among doctors with the whole evidence-based medicine announcements coming down from various task forces. In this morning's USA Today, for instance (right next to one of my op-eds!) there is a column from Dr. Marc Siegel at NYU complaining about the U.S. Preventive Services Task Force, which recently recommended against annual mammograms for women over age 40. Dr. Siegel writes that "Today I use my own checklist based on 20 years in practice." This involves sending women who are over age 40 for mammograms. He is sure that this is the right thing to do (and hence is writing an op-ed complaining about being told not to do it) but the reality is, based on the evidence, it isn't. That's exactly what the task force was "tasked" with figuring out. But he's a professional! He knows what's right based on 20 years in practice! Hence the title, "Task-force thinking doesn't deliver my kind of medicine."

The same thing, apparently, happens in teaching. When you teach for 20 years, you know how to teach, right? Certainly you know better than a scripted curriculum package... except that for the average teacher, that's not what this study seems to be saying. And just as doctors often object to standardization programs (for surgery, procedures, etc.) that try to turn them into robots, I don't think teachers like them either. But it raises the question in both areas: if the results are better (and there's no doubt that doctors following strict procedural guidelines make fewer errors), is it worth the cost? Nobody likes unhappy teachers or doctors. But we like results too.


Pablo A. Perez-Fernandez said...

An adaptive script should work even better in theory. That's why EPGY -- a robot, to use your metaphor -- is so effective for highly gifted kids. Furthermore, it is very effective for regular kids. Most people are unable to solve problems effectively. Who says freedom would allow them to teach well?

Jeremy said...

Two potential problems with the study (and implications of it):

1) Surely there were other differences between these three school types, and perhaps the students in them. I'm sure they tried to control for as many variables as possible, but this would seem to put too much weight on the instructional style.

2) Isn't it kind of obvious that if you're depending on standardized testing, that a very prescribed, scripted curriculum that exactly matches the content of the test will help students achieve better scores? My problem with that is that I don't believe these tests are very good measures of success -- perhaps for basic math and reading levels, but learning is so much more than that. Given enough time, a robot/computer/scripted teacher could train every kid to nail the standardized tests, but would that even be desirable?

Crissy said...

There are so many possible explanations. Maybe scripted lessons help underperforming teachers. Maybe the students benefit from the predictability of the lesson structure. Perhaps teachers who use scripted lessons, especially those who are talented and motivated, have more time to spend preparing real world examples or activities that help students grasp and enjoy what they are learning.