Friday, July 22, 2011

Tests and Cheating

We all like great stories. The seeming turnaround of the Atlanta public schools was one of them. Kids appeared to be doing better on standardized tests. Now it's come to light that principals and teachers were changing answers, with analyses of incorrect-to-correct erasures indicating astronomical odds against anything other than cheating going on. Even some Teach for America teachers have been questioned in all this.

Critics of NCLB and testing in general are claiming that this is the inevitable result of high-stakes situations. Hedge fund managers rewarded or fired based on performance may resort to insider trading. Athletes use performance enhancing drugs. I think these explanations go a little too easy on the culpable individuals involved, but I'm not particularly naive about human nature either. In situations where it is easy to cheat and cheating is rewarded, people cheat. By some estimates, about half of sole proprietor income (e.g. the guy who paints your deck) goes unreported in the US (and hence untaxed). Teachers judged on student test performance are subject to the same temptations.

I think there's plenty to criticize about NCLB -- not least of which is taking a focus away from high achievers who will always pass grade level tests, and concentrating teachers' time and attention mostly on children who are right around the edge of passing. Those children deserve attention, sure, but so do kids who are too far below grade level to catch up in the time any one teacher has them, and so do kids who will score in the 99th percentile.

On the other hand, in the absence of any accountability measures, you can wind up with kids graduating from high school who do not know how to read. Children are not necessarily served well by no testing either.

So how does one keep accountability while reforming some of the worst aspects of its unintended consequences? Long-time readers of this blog know I like a "value-added" assessment approach. Create tests that don't ceiling out at grade level (so you can get an accurate picture of gifted kids' needs as well). Then assess again throughout the year and judge teachers and schools on kids' improvement. In a digital era, this shouldn't be too difficult, and could in fact make testing fun. Frequent feedback can help teachers make spot changes (and is also more fair, given how much some students move around). Such assessments would reveal that a teacher who brought a fifth grader up from a 2nd grade level to a 4th grade level was in fact doing an awesome job -- and hopefully wouldn't put pressure on her to change a few more answers to get that kid up to 5th grade level. Such assessments would also show if a school's gifted students were merely treading water. Yes, they keep "passing" grade level tests. But that doesn't mean anything good is going on.


gasstationwithoutpumps said...

I like the idea of high-ceiling tests (and there are some available even now), but not the idea of frequent testing—American schools already waste over 5% of their time on taking and preparing for external tests, possibly more in weak schools.

Actually, more frequent testing would not necessarily be bad, as long as it removed all test preparation and replaced some of the current teacher-prepared assessment. But it is likely to have the opposite effect, of spending all of instructional time testing or preparing to test, with no actual instruction taking place.

Heather said...

I think assessments need to become more integral to instruction. The ideal would be where teachers and students could have a real time picture of progress.

I think curriculum developers need to look at how complex video games mechanics really work--because most video games are a series of tests, which reward kids with higher level tests--and we can't get kids to stop playing them.

The issue with video games right now is the content they test.

'Nother Barb said...

Our students, in addition to state-mandated tests, take a Measure of Academic Progress test twice a year. They take the test on a computer: each student starts slightly behind where he finished last time, and keeps on answering progressively advanced questions until he misses a certain number in a row.

Unfortunately, it is only used to determine the level of class he should be in; I don't think the school uses it to adjust curriculum or to add levels where needed.

As my son says, it is hard to prepare or cheat since every student starts at a different point. Unlike the state test, they do not turn in their "work" for math questions; they get scratch paper, but discard it on when they're done.

I wish "No Child Left Behind" was "Every Child Makes Progress".

Anonymous said...

The MAP has been a great disappointment in Seattle; it's not specific enough or reliable enough to give decent information, and it has a much lower ceiling than advertised. I like the original idea behind it, and was a great advocate of its being tried for a while, but now I am quite disillusioned (especially as it's vastly more expensive and time-consuming than it should be).

Anonymous said...

I am sorry, but I believe that if you are going to write on a topic you need to do the necessary research on said topic. For example, has anyone looked at any studies on cheating and why people cheat? I did, just out of interest, and the conclusion was that people cheated when the cost of cheating was less than the cost of failing. Not necessarily because they were hideous monsters.

Does anyone understand how difficult it is to raise a child up two grade levels in a single year, much less the 3 and 4 years you are suggesting? I have, and the statistics are staggering. That is what makes The Harlem School Zone so phenomenal because they have found a way to do it, but not with out huge cost as far as time of students and teachers, monetary cost to make sure that the child only has to focus on school and not where he is going to live, how they are going to eat, or if his mother or himself is in danger.

This group and the blogger of this site seems to be a very homogeneous group, which is fine, but I would expect that someone blogging as the expert on the subject would go outside of her comfort zone to bring the full picture back to her audience.

Anonymous said...

I am interested to hear the dissatisfaction with the MAP assessment. Our state with its new RTTT money decided to recreate a MAP-like assessment to be given 3x year and it clearly is inadequate for kids at either end of the curve - I don't understand testing methodology but with 50 questions it doesn't seem like the test can reach up(or down) to a level quickly enough to get a good measure for growth. I have tried to bring this up but feel that mostly I am being blown off by the powers that be who are focused on meeting grade level.

Anonymous said...

The idea of MAP is great, our schools start kids taking it in 2nd grade, 3 times a year. Teachers are supposed to prepare an Individual Education Plan for each child based on it. Sounds great, reality is my child spent 75% of her class time in 2nd grade reading books of her choosing. In 3rd grade her MAP scores started at the 6th grade level and dropped progressively throughout the year to the 4th grade level by year end. No one at school thought this was a matter of concern worth mentioning to me. The curriculum in 4th grade started to improve, and what do you know her test scores went right back up! I now ask the kids when they test so I can monitor the results.