A few folks commented on the "Are 5% or 10% Gifted?" thread about the wisdom of giving people a previous year's version of a test before an assessment. Is it a good idea or not? Some worry it could affect results; others point out that savvy folks will always find a prep copy of a test, so best to give it to everyone.
There's no doubt that familiarity with a test format -- and studying material that will specifically be on a test -- can raise scores. I was thinking about that when I came across this item from Wired News on a new SAT Prep video game. (As they point out, with a market this big, what took so long?) A few years ago, I received a review copy of one of the first of a new genre of SAT Prep novels (see this link for the SparkNotes version; lots of companies do these now). SAT vocabulary words are spaced throughout the book, with definitions in footnotes, so you learn while reading. Of course, kids have always learned new words by reading, but this satisfies that parental urge to make sure no time is wasted learning unimportant words... (sigh).
Anyway, there's no doubt that a kid who plays the SAT prep video game, and reads the SAT prep books, and takes a Kaplan course, will have some advantage over someone with no familiarity with the test. But here's something to think about: the vast, vast majority of people who use these aids do not score a perfect 2400 on the SAT (that number still looks weird to me, I still think in terms of 1600!)
Why not? If the test can be gamed through these tools, why doesn't everyone ace it?
The reality is that -- for all the prepping -- tests like the SAT still measure your ability to solve problems quickly when faced with new information. Even if you've seen 1000 math problems before, you haven't seen this exact one. You haven't seen this exact reading passage, so you haven't been asked these specific questions. For all their flaws, standardized tests still measure your ability to synthesize information and think on your feet.
For this reason, I don't think the test prep industry changes things as much as it might - either for the SAT, or for other tests. The fairest thing to do is probably just what NYC did -- make sure everyone has a copy, and then see what happens.