Tuesday, July 25, 2006

Those Crazy Statistics

The irascible Charles Murray is back today, writing on the op-ed page of the Wall Street Journal about black-white gaps in test scores achieved under No Child Left Behind.

Schools and districts are required to narrow the gap in test scores between various racial groups in order to receive certain levels of funding, or not be penalized. Of course, given the mean scores of different groups, and the number of people in each group, changes in mean scores can have very different effects on the gap, depending on where those means lie. Differences in pass rates, another key factor in NCLB tests, are also subject to the whims of statistics put in the hands of people who have a vested interest in seeing them go one way or the other.

For instance, Murray quotes Pres. Bush's statements about Texas. There, 73% of white students passed the math test in 1994, while only 38% of African-American students passed it. In 2003, Bush was happy to note, the gap was only 10%. That seems great, indicating that African-American students have made a great deal of progress -- more than their white counterparts. But it's somewhat misleading. Say in one state, the pass score is relatively low -- where many students will be able to achieve it. If one group of students has a higher mean score than another, raising each group's mean score by, say, 10 points, will naturally narrow the pass-rate gap by more than that (if the scores follow standard bell curve distributions). On the other hand, if a pass rate is set particularly low in another state (say, only 40% of all students will pass), a gain in everyone's test scores could in fact make the gap appear worse, Murray notes.

Both districts under that scenario will have improved test scores. But because the second district will appear not to have closed the gap between groups, it could suffer penalties, while the first one will be rewarded.

Studies that have come out earlier this year about state NCLB tests have found that indeed, most states are making them ridiculously easy to pass in order to make sure that pass rates are high and racial gaps appear to narrow. Given how states are citing NCLB as a reason to move away from gifted education to focus on improving everyone's scores, it would be nice to get slightly more meaningful data. As Murray notes of NCLB, in practice, "It holds good students hostage to the performance of the least talented, at a time when the economic future of the country depends more than ever on the performance of the most talented."

No comments: