Friday, December 12, 2008

Teaching's "Quarterback Problem"

Malcolm Gladwell, author of Outliers, has an article in this past week's New Yorker called "Most Likely to Succeed." He weaves together two narratives -- the problem a NFL scout named Dan Shonka faces in trying to advise teams on which quarterbacks to draft -- and the problem of figuring out how to hire effective teachers.

The link? Top-drafted quarterbacks have an incredibly mixed record in the NFL, in part because the game is just so much more advanced than what they play in college. Rather than throw a perfectly straight pass to an open receiver you can see, you have to throw a perfectly straight pass to a well-guarded receiver you can't see. And you'll be doing this while 4 300-lb guys are trying to dive on top of you. You can't judge a college quarterback on how well he executes the same play over and over; in the NFL he'll have to improvise more because the defense is far less predictable. "There are certain jobs where almost nothing you can learn about candidates before they start predicts how they'll do once they're hired," Gladwell writes. "So how do we know whom to choose in cases like that?" The "case like that" with the most profound social consequence, of course, is teaching.

Readers of this blog know that teacher quality really matters. We've discussed Eric Hanushek's work at Gifted Exchange before, which shows that kids assigned to a good teacher and bad teacher can diverge wildly in achievement in as little as a year. In three years, it's an almost insurmountable difference. As Gladwell points out, "teacher effects dwarf school effects: your child is actually better off in a 'bad' school with an excellent teacher than in an excellent school with a bad teacher. Teacher effects are also much stronger than class-size effects. You'd have to cut the average class almost in half to get the same boost that you'd get if you switched from an average teacher to a teacher in the eighty-fifth percentile."

If you talk to any principal she can tell you -- quite easily -- who's an effective teacher at her school and who isn't. You can also see it in student test scores. A few districts test students at the start of the year and the end of the year. Barring massive differences between classes, this gives you an idea of which teachers give the most "value-added" bang for your buck. If you tested effectively, you could even judge teachers of gifted kids this way -- the kids might score at the 99th percentile on grade level tests, but give them an out-of-grade level test. I recall something like this happening in 3rd grade for me. I took the 3rd grade standardized math test, got a 99, so I took the 4th grade. I got a 99th percentile on that too, so I took the 5th grade test. My score there? 79th percentile -- which meant this was probably the level where I belonged. By the end of the year I was at the 99th percentile on the 5th grade test. That 20 percentage point jump is a pretty good indication I was learning something.

But how do we define effectiveness? Or is it just like the famous quote about pornography, that we just know it when we see it? Gladwell discusses watching tapes with folks from Virginia's Curry School of Education to show how effective teachers behave in the classroom -- they tend to engage students, evaluate where everyone is, problem solve to figure out new ways of explaining things, and keep order by redirecting kids who are misbehaving. There really are big differences -- he describes one trig teacher who, in the time it takes a less effective teacher to boot her computer because she forgot to turn it on, has managed to interact with every child in his math class.

But while it's obvious to everyone which practicing teachers are good and which aren't, Gladwell says, it's not obvious to anyone when you're hiring a new teacher who will be good and who won't. The things we typically look for -- masters degrees, teaching certificates -- tell you absolutely nothing. Only being in front of a class can tell you that. But by the time a teacher has her grounding in front of a class -- 2 years in -- she's protected from firing by a rigid tenure system if she's bad, and rewarded less than a 30-year piece of deadwood if she's good.

So Gladwell suggests changing up the process. "We shouldn't be raising standards" for hiring teachers, he says. "We should be lowering them, because there is no point in raising standards if standards don't track with what we care about. Teaching should be open to anyone with a pulse and a college degree-- and teachers should be judged after they have started their jobs, not before. That means that the profession needs to start the equivalent camp. It needs an apprenticeship system that allows candidates to be rigorously evaluated... you'd probably have to try out four candidates to find one good teacher. That means tenure can't be routinely awarded...An apprentice should get apprentice wages. But if we find eighty-fifth percentile teachers who can teach a year and a half's material in one year, we're going to have to pay them a lot -- both because we want them to stay and because the only way to get people to try out for what will suddenly be a high-risk profession is to offer those who survive the winnowing a healthy reward."

(As a side note, this tracks my point about treating the teaching corps like McKinsey in my Michelle Rhee post: McKinsey is more selective than most big school districts, but it still hires a lot of associates and tries them out. Most leave of their own accord or are "counseled to leave" in three years. It's hard to stick around to make partner. But once you do, the rewards get much better).

It's an interesting piece and, in Malcolm Gladwell fashion, very engaging. Of course, he paints with a broad brush to make his point (also vintage Gladwell). Though Gladwell doesn't mention this, Hanushek studies have found that one or two variables we can figure out beforehand do matter some. A key one? Teachers' scores on standardized tests. That means that, other things being equal, a teacher who scores a 700 on the SAT math section is going to be a more effective teacher than one who scores a 500. The higher score tends to indicate that the teacher is better able to figure things out quickly. This ability to solve problems quickly is a key component of the "withitness" that Gladwell notes is a common attribute with good teachers. (Perhaps it's another way of saying "intelligence" but since Gladwell just wrote 2 chapters in his last book about how IQ doesn't matter, he doesn't mention this).

Furthermore, Gladwell gets so excited about proving a point that he often fails to be consistent across all his writings, arguing different points with equal vigor. For instance, in Outliers, he reprints charts showing that poor students in Baltimore learn as much as better-off students during the school years, it's just that they learn nothing (or forget things) over summer break. All the talk of reforming schools "assumes that there is something fundamentally wrong with the job schools are doing," he writes. "But look back at the second table, which shows what happens between September and June. Schools work. The only problem with school, for the kids who aren't achieving, is that there isn't enough of it."

Or is the problem that they have lousy teachers? Many more hours over the summer with a lousy teacher is not going to help matters.

The point is that there are many things that need addressing with education. For instance, even bad teachers can do better in classes that are tracked by ability than in heterogeneous classes. We need a lot more acceleration, teachers with better test scores, a more rigorous curriculum, reforms to tenure, etc.

But Gladwell does raise interesting points to a broad audience, and so I'm glad to see this piece in the New Yorker. A mediocre education system isn't good for anyone. Teacher quality matters a lot, and with Bill Gates and others now focusing on it, we may actually get somewhere.

1 comment:

Jeremy said...

Wow, fantastic post -- the original article is great too, but your analysis makes it even more interesting.