Tuesday, April 29, 2008

The SAT Prep Game

A few folks commented on the "Are 5% or 10% Gifted?" thread about the wisdom of giving people a previous year's version of a test before an assessment. Is it a good idea or not? Some worry it could affect results; others point out that savvy folks will always find a prep copy of a test, so best to give it to everyone.

There's no doubt that familiarity with a test format -- and studying material that will specifically be on a test -- can raise scores. I was thinking about that when I came across this item from Wired News on a new SAT Prep video game. (As they point out, with a market this big, what took so long?) A few years ago, I received a review copy of one of the first of a new genre of SAT Prep novels (see this link for the SparkNotes version; lots of companies do these now). SAT vocabulary words are spaced throughout the book, with definitions in footnotes, so you learn while reading. Of course, kids have always learned new words by reading, but this satisfies that parental urge to make sure no time is wasted learning unimportant words... (sigh).

Anyway, there's no doubt that a kid who plays the SAT prep video game, and reads the SAT prep books, and takes a Kaplan course, will have some advantage over someone with no familiarity with the test. But here's something to think about: the vast, vast majority of people who use these aids do not score a perfect 2400 on the SAT (that number still looks weird to me, I still think in terms of 1600!)

Why not? If the test can be gamed through these tools, why doesn't everyone ace it?

The reality is that -- for all the prepping -- tests like the SAT still measure your ability to solve problems quickly when faced with new information. Even if you've seen 1000 math problems before, you haven't seen this exact one. You haven't seen this exact reading passage, so you haven't been asked these specific questions. For all their flaws, standardized tests still measure your ability to synthesize information and think on your feet.

For this reason, I don't think the test prep industry changes things as much as it might - either for the SAT, or for other tests. The fairest thing to do is probably just what NYC did -- make sure everyone has a copy, and then see what happens.

Friday, April 25, 2008

Ability grouping may matter most for gifted black kids

While the national obsession with testing is often frustrating for teachers, students and parents alike, it does have a silver lining: We are now awash in data on student performance.

These data points are showing some interesting things. According to a recent article in Education Week (go to Edweek.org, then search for "Black-White Gap Widens Faster for High Achievers;" the URL is too long to type easily in Blogger), a series of papers presented at the American Education Research Association meeting this year showed that the achievement gap between high-scoring black and white students grows faster as these children progress through school than the gap between lower-scoring black and white students.

The results are pretty clear. The first study Ed Week mentions, by Sean Reardon, a professor of education and sociology at Stanford, analyzed the findings from the 7,000-student Early Childhood Longitudinal Study (which tracked kids from kindergarten to fifth grade). Reardon found that the black-white achievement gap grew twice as fast among students who scored above the mean as it did among those who scored below the mean. (A link to a PDF of the paper is included from the Ed Week site).

The second study, from economists Steven Rivkin and Erik Hanushek (incidentally, one of my personal favorite researchers), tracked 800,000 Texas children from 3rd grade through 8th grade. They grouped the children into four quartiles based on 3rd grade reading scores, and then compared later test results. They found that the higher the initial scores, the more the scores between black and white students diverged on later tests.

A third study, by Harvard researcher Lindsay Page, sheds a bit of light on these findings. She discovered that differences between schools, as opposed to within schools, contributed more to the black-white achievement gap in recent years. As she noted, "the cost in achievement to black students from attending racially segregated schools increased over the last three decades of the 20th century."

What does this mean? It's actually an important point. It's fashionable to lament that in schools with extensive ability grouping, white students are over-represented, and black students are under-represented, in the top tracks (for an example of one such lament, see this op-ed from doctoral student Beth Rhodes that ran recently in the Roanoke Times). But the reality is that, over the past 30 years, there has been almost no change in the proportion of black and white students actually attending the same schools in order to be unfairly transferred into certain tracks in the first place.

According to Page's study, between 1971 and 1999, the average white student attended a school that was 7-8% black, and the average black student attended a school that was about 60% black. For a variety of reasons -- too many to get into here -- Americans have largely maintained patterns of residential segregation, which then correspond to school segregation. Given recent Supreme Court decisions that limit circumstances in which schools can take race into account for enrollment, this is unlikely to change soon.

(I do want to add, though, that the issue is more complicated than people often think. Last year I had the pleasure of visiting Louisville's Central High School, a fairly strong school which has long played a prominent role in the city's African American community. In the midst of Louisville's desegregation push, Central was required to limit its enrollment to 50% black students, which required it turn down many neighborhood kids, kids whose parents had fond memories of Central, kids drawn to its special programs, etc., all because these kids were black. But the school had to take any white kid who walked in the door. A judge eventually said enough is enough and let its enrollment reflect the make-up of its application pool).

Anyway, Central High School's reasonable results aside, many predominantly black schools are plagued by poor discipline, bad teachers, low expectations, and have a high proportion of students whose families do not particularly value education. These schools often look less like safe, orderly Central, and more like Malcolm X. Shabazz High School in Newark, whose metal detectors and guards, one parent once told me, made the place seem harder to get into than an airport. At Shabazz, in 2007, only 14% of 11th graders passed the math section of New Jersey's High School Proficiency Assessment, which is required to graduate.

For a variety of lamentable reasons, the average academic performance at an average predominantly black school is lower than the average academic performance at a predominantly white school. This affects all children, but it affects one particular group in a particularly horrible way: gifted black students.

As Reardon notes, "the average black kindergarten student with a given level of math or reading skill attends a school with lower mean cognitive skill than the average similarly-skilled white kindergarten student. Initially high-skilled black students will be in schools where they are farther above the median student than similarly-skilled white students. If curriculum and instruction in schools are tailored to the median student in the school, then high skill black students will, on average, receive less challenging curriculum and instruction than their similar white peers, leading potentially to differential rates of achievement growth between such students."

This is a critical point. People who complain about racial disparities in ability-grouping within schools are talking about integrated schools. The majority of black students do not attend highly integrated schools. In an all-black school, it should go without saying that all the tracks, high to low, will be all-black.

And it turns out that creating those tracks may be more important than anyone previously realized. A student who scores at the 98th percentile on tests, who is forced to attend classes aimed at the 40th percentile day-in and day-out, will become frustrated and bored. She surely won't learn as much as she could in a class aimed at the 90th percentile. Would it surprise anyone if, two years hence, her test scores have fallen in comparison to those of a white student also attending a non-tracked class that's, nonetheless, aimed at a different median score?

As the Davidsons and I wrote in Genius Denied, "Foisting 'equality' to the exclusion of excellence on schools hurts bright children, but it does not hurt all bright children equally." Better-off, better-connected parents can make other arrangements when the public schools fail to challenge their gifted kids. Gifted minority kids, and gifted poor kids, are often stuck with the schools they get.

That's why gifted education is most important for these kids. "They don't deserve to be abandoned to boredom and underachievement to satisfy someone else's agenda," we wrote. These new studies show just how dangerous that abandonment truly is.

Thursday, April 17, 2008

NYC debates: Are top 5% or 10% gifted?

New York City has made a number of positive changes to its gifted programs recently. In the past, different districts had different standards, and the application process was far from standardized. This meant, inevitably, that more savvy parents were better able to navigate the system, and that being gifted meant something different in some districts than others. This year, all children were able to take a 2-test screen for giftedness, and anyone who scored in the top 5% (on a national scale) would be guaranteed a seat in a gifted program.

This was as fair and straightforward a way of assessing giftedness as possible. Using two tests enabled scorers to look at multiple data points, and since the two tested different things (one aimed to be more of an intelligence test, and one more of a school readiness test) the tests were supposed to give a more complete picture.

Unfortunately, the results didn't turn out the way many people in the city wanted. This article from the New York Times explains that only 1,637 of about 50,000 kindergarten and first grade applicants met the 5% cut-off. Currently there are 4,649 kindergarten and first grade children enrolled in the city's gifted programs.

Not only was the overall number lower than people expected, some districts didn't even produce enough top scorers to field a single full class. District 7 in the South Bronx, for instance, had only 5 children score above the 5% cut-off.

So what did the New York City schools do? To their credit, they didn't revert to the old standard whereby the definition of giftedness changed from district to district. Instead, they changed the definition of giftedness overall. Now, students who score in the top 10% on a national scale on the two tests will be eligible for a seat. This raises the total number to about 3,000, and produces a full class of 13 children in the South Bronx (even so, District 16 in Bed-Stuy will have only 5 children qualify under the 10% standard; last fall 34 children enrolled in gifted programs. Previously enrolled children are not affected by the new standard).

What can I say? There is no magic number that makes someone gifted. Ten percent is no more arbitrary than five. I think 10% is politically more difficult to defend than interventions for the top 0.1% kids who cannot be served in normal classroom environments. Many kids in the top 10% can be served in rigorous, regular school programs as long as they're ability-grouped to some extent. Or they can skip a grade and be served well in the next class up. The problem is that New York does not, as a general rule, have a lot of rigorous schools or grade skipping. So gifted education winds up filling this gap. Since they had the teachers, had the seats and had the budget, they choose to fill the programs, rather than re-jigger the whole concept.

What do you think, readers? Did New York City do the right thing?

Monday, April 14, 2008

The Alleged Engineering Shortfall

This morning's Wall Street Journal op-ed page Notable & Quotable section contained an interesting excerpt from education reporter Jay Matthews' current Wilson Quarterly article on US schools and the US economy.

It's no secret that American schools are not producing as many top-rated engineers, mathematicians, physicists, computer scientists and the like as our economy requires. However, Matthews points out that perhaps our statistics on the engineering gap are, dare I say it, more evidence of our problem with math.

Article after article (and the US Dept of Education) has noted that China turns out 600,000 engineers a year, and India 350,000. The US? Just 70,000. However, the China numbers come from the China Statistical Yearbook, a government publication which seems to be exaggerating. As Matthews notes, a later McKinsey Global Institute report found that about half of those engineers would be considered mere technicians in the US. The 350,000 in India figure -- originally calculated by Fortune magazine -- has not been reproduced, and the NSF claims it's likely lower.

A 2005 Duke report, apparently, found that the US graduates 137,437 engineers with at least a bachelor's degree a year. India cranks out 112,000, and China 351,537. Yes, that Chinese number is higher, but China's population is well over three times that of the US. A Washington Post article quoted educational psychologist Gerald W. Bracey as saying that "That's more U.S. degrees per million residents than in either other nation."

The point is not that the US shouldn't be concerned about our supply of engineers. My guess is that American colleges are cranking out more "recreation" or "leisure" studies majors than top-notch engineering graduates.

But those of us who write about education should be careful not to use hyperbole when we don't need to. The truth is, a lot of the talk about losing our scientific edge amounts to concerns about how immigration has changed. These days, top scientists may come to the US for a while, get a green card, and then move around between the US and their home countries in a more circular fashion. This is a trend in the internationalization of science, and needs to be discussed in its own right, rather than relying too much on national numbers (though they do look good in op-eds!)

Wednesday, April 09, 2008

New Jersey Governor's School... RIP?

For the past two years we've been following the saga of New Jersey's Governor's School. Every summer, this program allowed the state's brightest high school students to attend one of several programs (in the arts, engineering, sciences) free of charge. The programs allowed kids to meet other kids who shared their interests, spend the summer doing something productive, do "real" lab work or work with professional musicians, etc. Historically such programs (which exist in at least 17 states) have been free of charge in order to draw kids from all backgrounds. Many families do not have the disposable income necessary to pay for multi-week camps, particularly if there are multiple children in the family. They may not necessarily be poor (in which case other scholarships for summer camps may become available) but they are not well-off enough to afford the $5,000 or more a month-long camp could cost.

A little over two years ago, New Jersey decided that -- given the limitations that are always a factor in any public budget -- Governor's School would go on the chopping block. Alums and state luminaries who were alarmed by all this quickly raised enough money to keep the programs open for the summer of 2006.

Unfortunately, this temporary reprieve has been just that: temporary. Two years later, Governor's School is still largely out of the state budget. Many of the original large corporate donors have moved on. They gave money initially because of personal connections, but their larger mission is not the education of bright children, so ongoing gifts do not make sense for them. The Robert Wood Johnson Foundation, for instance, gave $652,000 in 2006, but nothing in 2007 or 2008. Sovereign Bank gave $200,000 in 2006, but nothing in 2007 or 2008. Since the Governor's School Board has not been able to raise enough private money during the budget limbo, they are scaling back programs considerably. GS Arts will be 10 days instead of longer. GS Engineering requires a longer project, so it will cut slots (possibly to 40 from 100).

People can certainly quibble over whether the state should be paying for special summer programs for bright children (though it pays for remedial summer school). But what's particularly fascinating is that New Jersey is a very wealthy state. Most other states with Governor's Schools are not nearly so wealthy. And yet, they still support most or all of their Governor's School projects with state funds. Arkansas and West Virginia, for instance, fund 100% publicly. Even Louisiana, still recovering from Hurricane Katrina, funds more than 50% of its Governor's School out of state funds.

One can only assume that the New Jersey powers-that-be have decided that all smart kids are rich, and hence their families should pay for any additional education they think they need beyond the 180 days a year the state allots them in their local public schools. It's an interesting theory. Anyone who's met extremely bright kids knows it's false, but unfortunately, the middle-class status of most works against them. They're not well off enough to afford private summer schools, but not bad off enough to attract corporate philanthropy or government largesse that seeks to target the "underprivileged." The result is the decline of an institution that once made New Jersey proud.

Friday, April 04, 2008

More on Smaller Class Sizes

Given that class size reduction is a key (and costly) component of many school reform efforts, I'm happy to see that it's being studied extensively. In the past few weeks, we've looked at a study from Kenya that showed that class size reduction is most effective when accompanied by ability grouping, and a rehash of the Tennessee STAR study that found that smaller classes benefit higher-achieving students more than lower-achieving students.

Now, a new study looking at schools in four countries (US, UK, Hong Kong and Switzerland) asks another question. If we assume that small classes are better for kids (which, I want to stress again, has not been proved conclusively -- it is very hard to isolate variables), why would we suppose that would be? Is it because teachers teach differently when they have smaller classes? Or is it because students behave differently in small classes?

According to research presented at the American Education Research Association conference in late March (and written up in this article in USA Today), it's not the former. In small classes, many teachers continue to teach as though they still have a large class. They make presentations that don't necessarily involve the students; they don't demonstrate what the students themselves will then do. But the students themselves spend more time on task. It seems that teachers are better able to maintain control in smaller classes, and so more time is spent on learning, and the students are able to spend more time engaged with the teacher than they are in larger classes.

It's a tricky issue. Perhaps if teachers were trained to better leverage their smaller classes, then small classes would result in more benefits. But it's hard to know. The article notes that in two of the four countries studied, the results were inconclusive. Indeed, some countries with the best academic results have the largest classes! This article (and chart, about halfway down) shows that while we quibble over whether classes have 20 or 25 pupils, in South Korea, Singapore, Hong Kong and Japan (high scorers all), class size averages over 30 students.

My own opinion? From what I've seen of various studies, I'm currently thinking that teacher quality and (much harder to control) student quality matter most. By "student quality" I mean kids coming to school ready to learn, well-fed, well-rested and safe, and firmly ensconced in a culture that values education. After that, ability grouping is probably the most important factor. After that, class size. So yes, we can spend billions to make classes smaller. But paying brilliant teachers $150,000 a year and removing any teacher who can't keep up is probably a better idea, even if it means class sizes of 30 or more.

Tuesday, April 01, 2008

Reducing Class Size May Increase Learning Gap

Ever since the first battle over passing No Child Left Behind, there's been a lot of discussion about closing the achievement gap between children of higher and lower socioeconomic status in schools. It appears quite possible that Congress will not reauthorize the bill. While this won't be the end of frequent testing and accountability as a way to close the achievement gap, people will definitely be throwing out other new ideas as well.

One popular one is the idea of reducing class sizes in the early grades. The oft-cited Tennessee STAR study, which randomly assigned thousands of young children to smaller or larger classes, found that children in general achieved better test scores when they were assigned to smaller classes. Other studies have cast some doubt on this finding, but for purposes of argument, let's say it's true.

Here's the thinking: children of lower socioeconomic status often arrive at school without a lot of the school preparation middle class families take for granted. Kids and parents may not necessarily read together. Kids might not know letters and numbers. Intuitively, it seems like giving the less-prepared kids more time and attention from teachers -- in smaller classes -- might help them catch up.

But this turns out not to be the case. According to a new look at the STAR study by Northwestern professor Spyros Konstantopoulos in March's Elementary School Journal, smaller classes do boost all students' achievement levels, but they also increase the achievement gap. It turns out that smaller class sizes benefit high achievers more than lower achievers. So while all students do better, high achievers do much better, and lower achievers do a little better.

It's not clear why this would be, but it complicates the "small class size" argument. Since we know that reducing class size and tracking by ability increases test scores more than simply reducing class size alone (see the Kenyan study from a few weeks ago) it's possible that ability grouping could help close the achievement gap. But perhaps not -- one would have to test all these variables at the same time.

Personally, I think it's more important that everyone do better than that you close the achievement gap, but not everyone thinks that way. For instance, many politicians and others lament the "growing gap between the rich and poor" at every opportunity, even though almost everyone in the US is better off materially than they were 50 years ago. Sometimes people view relative position as more important than absolute position, and so we will see how the education community interprets these latest results. Anyway, it's food for thought.