In his Patch essay of November 1 (“The Deleveling Debate — How Wrong Questions Lead to Wrong Answers”), Bill Tally takes issue with Jeffrey Bennett’s op-ed, “Deleveling’s First Year — Lower Scores, Grade Inflation, and Spinning the Truth.”  Tally labels Bennett’s analysis of preliminary outcomes from the school district’s 7^th grade deleveling experiment “alternate spin”—an odd characterization, unless what he meant to acknowledge was the original spin of Superintendent Brian Osborne’s report to the community, in which he judged deleveling a “promising” success. Or, for that matter, Dr. Osborne’s report last year citing studies that support deleveling and ignoring those that don’t. Osborne will almost certainly announce at the December BOE meeting his intention to delevel eighth grade in the coming year, and possibly ninth. Before that happens, would it be too much to ask him—or anyone in authority—for a more realistic appraisal of the results of deleveling thus far? 

Tally does not offer one. He begins by reciting a by now familiar catechism: “The leveling system was rigid, and it was virtually impossible to break out of lower levels and successfully move up to the true college prep classes, mislabeled ‘honors classes.’ As early as age eleven or twelve, children were being permanently channeled into a path to nowhere.”

But in the first place, these declarations about a “path to nowhere” assume that leveling, and not any of several other likely causes, is what prevents children from succeeding in middle school, high school, and ultimately college—a claim that rests solely on correlative evidence. Indeed, Dr. Osborne’s observation that “students participating in higher level classes show an increased college graduation rate” reminds me of the now discredited belief that hormone replacement therapy protects women from heart disease. Doctors had, after all, observed a correlation between HRT and better cardiovascular health. But once HRT was subjected to controlled studies, it turned out it actually increased the risk of heart disease. What doctors thought they saw at first was only an artifact of their tendency to prescribe HRT to wealthier, healthier women less predisposed to heart attacks and strokes.  

Second, although level assignment has undoubtedly been “broken” for many years—specifically with respect to the inflexibility that Tally rightly decries, and especially for kids with behavior issues and learning or other disabilities—his claim that moving up has been “virtually impossible” is not true. In 2009-10, for instance, nearly 20% of students assigned to Level 3 Science in seventh grade were recommended for Level 4 in eighth grade, as were more than 23% of Level 3 Language Arts students and 28% of Level 3 Social Studies students. Yet only about 11% of Level 3 students earned A’s in their course work for Science, 10% earned A’s in Language Arts, and 9% earned A’s in Social Studies. Deleveling advocates tell us that Level 3 classes can’t be characterized as college prep. But many 7^th graders earning below an A in this ostensibly unchallenging stratum obviously were recommended for Level 4 in every subject. So was it really “virtually impossible” to move up?

There undoubtedly are children in our district who should move up, and don’t. But isn’t it also possible that at least some kids who don’t move up, shouldn’t?  

Comparisons on Shifting Ground

Mr. Tally cites as evidence of the success of deleveling in 2010-11 (“leveling up” in Dr. Osborne’s nomenclature) the fact that following the 7^th grade experiment, about half the leveled-up 7^th graders went on to Level 4 in 8th grade. But this wasn’t because all those students’ performance improved. As Mr. Bennett has shown in an analysis of pre and post “level-up” assignments (posted to Maplewood Online) the administration simply altered its standards. In 2009-10, the year before deleveling, a seventh grader in Level 3 who earned at least a low B on the Science final exam could be recommended for Level 4 Science in eighth grade. But in 2010-11, the year of deleveling, seventh graders needed only a low C on the final to be recommended for eighth grade Level 4. Not surprisingly, more than twice as many kids—nearly 47%—were recommended for Level 4 Science that year. This same pattern held for Social Studies, where only 28% of Level 3 students were recommended for Level 4 in 2009-10, but more than half (53%) got the nod in 2010-11.

Is there anything wrong with encouraging more kids to try their hand at a more demanding pace or depth of instruction? On the contrary, there’s everything right with it, if moving them up has a good chance of helping them succeed. But deleveling does not “level up” only those who will be likely to pull at least a B in Level 4. It also puts kids into Level 4 who will fail there, or become discouraged struggling to keep their heads above water.

Comparing final exam grades is helpful to see some of what actually happened in 2010-11, because the final has stayed the same from year to year and, as a district-wide exam, may be less subject to grade inflation than course grades. On the Science final, Level 3 students who were “leveled up” did worse in 2011 than Level 3 students in 2010’s leveled classes, earning only about a quarter as many A’s and 14% more C’s. Only the percentage of B’s, D’s, and F’s remained comparable between the leveled group in 2010 and the deleveled group in 2011. Moreover, Level 3 students in both years did poorly on the exam overall:  45% in 2010 scored a D or an F and almost 44% in 2011 scored a D or an F. Figures like these suggest we’ve got a lot of kids for whom “leveling up” won’t be the main factor that determines whether they excel in high school, get into college, or leave with a degree.

So when we ask, “Did the students originally recommended for Level 3 benefit from the change?” the answer, in Science at least, was “No,” for around half of them. But some of the rest—kids who were able to rise to the challenge and earn B’s and A’s—surely benefitted. Why, then, couldn’t the criteria for Level 4 placement simply have been relaxed for more students in 7^th grade last year, instead of deleveling the class altogether?

It’s somehow not reassuring when Mr. Tally, who sometimes seems to be channeling the voice of the Superintendent or the Board President, responds to such data as the 23% increase in the percentage of leveled up kids failing Social Studies, and the 25%  increase in those failing Language Arts, and the more than doubling of the percentage of leveled up kids failing Science, by writing: “There is no reason to be surprised by this, however, or see it as harmful, since leveled up students were taught and graded on a more rigorous curriculum than level 3 students were the prior year.”

No reason to see it as harmful?

More course failures sound pretty harmful to me, especially since poorer final exam scores suggest that many of these students not only struggled more: they learned less. If, as Tally says, “Twelve years old is far too young to be told that you are destined to fail,” then surely it’s also too young to be shown that it’s just too hard to succeed.

What Happened to Closing the Achievement Gap?

That brings us to the question of the achievement gap, and what effect deleveling is having for Black children in our community. It really deserves emphasis that for high performing African American kids, the achievement gap with whites almost doubled in Language Arts from 2009-10 to 2010-11, even as our district’s average scale scores in Language Arts remained essentially unchanged as a percentage of our DFG’s scores over the same period. (SOMSD reached 97.2% of the DFG average score for 2009-10 and 96.8% of the DFG average in 2010-11.)

For kids scoring advanced proficient on the NJASK, there was a racial gap of about 17 points in 6^th grade (2009-10), which rose to about 29 points in 7^th grade (2010-11). The number of Black kids scoring advanced proficient was very low in both years; most of the gap increase was actually caused by whites leaping ahead in 7^th grade (from about 21% to about 35% advanced proficient). But 2010-11 wasn’t just a bad year. It was the culmination of a three year decline, both in the absolute percentage of Black 7^th graders scoring advanced proficient, and in their advanced proficient percentage increase from 6^th to 7^th grade each year. So, for deleveled Black 6^th grade students transitioning to leveled 7^th grade in 2007/08 - 2008/09, there was an almost 9-fold increase in the percentage of advanced proficient scorers (0.9% to 7.9%). For those transitioning in 2008/09 - 2009/10 (again, deleveled to leveled), there was a greater than 4-fold increase (3.3% to 14.7%). Then, in the 2009/10 -2010/11 transition, the bottom fell out: the percentage of Black advanced proficient scorers increased from 4% to only 5% when they moved from deleveled 6^th grade to deleveled 7^th grade. And this happened while white students were going from scoring nearly 30% advanced proficient in sixth grade to scoring nearly 35% in seventh.

Does this trend mean 7^th grade deleveling was a failure for advanced proficient Black kids? Not necessarily. Maybe it just means talented Black students are leaving the district after 6^th grade.

Please Make Up Your Minds

According to Mr. Tally, “We cannot determine the success or failure of deleveling by looking only at scores on standardized tests. We must look at multiple data points in order to develop the best picture of the outcomes.” He’s quite right. But it has always been clear, regardless of his declaration that deleveling “was never promoted as a silver bullet that would single-handedly conquer the achievement gap overnight,” that deleveling alone would be credited for the success of any reforms that might accompany it. In fact, deleveling’s supporters have always justified its necessity by painting leveling as the most relevant cause of every ill associated with the achievement gap: damaged self esteem, inhibition of achievement in high school, student attrition in college. Deleveling was portrayed as the fundamental cure for these ills—the necessary prerequisite without which student supports, professional development, and after-school programs, etc., could have no real impact. But when objective test scores for deleveled students show either trivial differences or a qualitative decline for those deleveling was meant to help most, its advocates can’t really turn around and protest that deleveling was never meant to be a silver bullet. Anyway, how will we discover which bullets are hitting their targets? If the district’s new (and welcome) early interventions, greater focus on core academic subjects, and curricular revisions were having a wonderful effect on students’ progress and yet deleveling was undercutting those reforms’ effectiveness in some way, would we have any way of knowing it?

What Do You Call Harm?

Deleveling’s detractors have always feared it would be hastily exonerated if it turned out to cause problems its supporters believed could be disguised or ignored. And sure enough, advocates like Tally have emerged to claim there’s “no evidence” that those recommended for Level 4 have been harmed by deleveling. Curiously, for someone who denies that test scores alone measure everything of value, Mr. Tally rests his argument about lack of “harm” on the percentage of kids deemed Advanced Proficient on the NJASK. Suddenly, test scores are all we need: not measures of the amount of students’ time that is wasted completing assignments appropriate for far younger children, or the frustration they encounter in classes where the pace is too slow, or the opportunities they are never offered because their achievement isn’t interesting to social crusaders. 

The problem with the NJASK as a measure of genuine “honors” level proficiency is that it is designed to indicate median achievement, flag low achievement, and provide a “finish line” estimate of greater proficiency. “Advanced proficiency” is attained when a student scores 250 out of 300 on the exam. Percentage distinctions within the Advanced Proficient range are never publicly reported. In fact, the state’s interpretive documentation warns that:

“Scores on the NJASK tend to be…less precise at the extremes, so the accuracy of score differences in the vicinity of 200 tends to be greater than in the lower part of the partially proficient range or the advanced proficient range. This point is of particular significance for the use of scale scores to identify students for placement into advanced or honors classes, as more latitude and flexibility is called for in interpreting scores in that part of the score distribution.”

In other words, the NJASK cannot reliably measure changes at the highest end of student achievement, so it will not tell us whether those students are “harmed.” Happily, Mr. Tally acknowledges Mr. Bennett’s revelation of coursework grade inflation last year, and concedes it “may suggest a need for further differentiated instruction for the more advanced students to keep them engaged and challenged.” But does he really believe what passes for differentiated instruction in middle school now—e.g., extra assignments providing no greater depth, and group work of dubious academic value—actually rises to the standard of a free and appropriate education for advanced students? There are quite a few parents who would argue otherwise.

How Passionate Advocacy for Deleveling Leads to Promises That Can’t Be Kept

Tally’s final bullet points assert that the first-year results of deleveling have been “encouraging,” and declare that “Deleveling provided opportunities for growth to students who would have been harmed under the previous system.” But as the increase in course failures among deleveled kids indicates, deleveling also caused some students to learn less.  If “many…used the opportunity well and are better positioned for future success as a result,” others got left behind. If those students’ needs are eventually met, will others’ needs be left behind?

It’s easy to see why they might. For if there’s “no evidence that students originally recommended for Level 4 were harmed by deleveling,” then isn’t Mr. Tally (or Dr. Osborne, or whomever he speaks for) just throwing parents a bone when he mentions their complaints about advanced children not being adequately challenged? When he admits, in thoroughly passive language, that “More needs to be done … to ensure challenging work is provided … including [for] the most advanced students,” one gets the sense that this action item has not quite found its way onto the district’s administrative agenda.

If the administration continues to marshal evidence to insist that no harm is done by deleveling, then how will things like offering advanced students more challenge ever become prioritized? If, on the other hand, we can admit that some students—both at the top and the bottom of the achievement curve—are harmed by deleveling, then shouldn’t we remain open to other pedagogical strategies, and take care to ensure deleveling doesn’t become an end in itself?  It’s not impossible that the tradeoff with lowered self esteem for struggling students is, as delevelers argue, a better education—at least for some kids. But what’s never explained is why there must be a tradeoff at all: Why isn’t it preferable to redesign leveling so that enhanced curricular rigor and support in lower-level middle school classes provides a sheltered period—perhaps one even administered quarterly—during which the objective is to prepare Level 3 students for the rigors of Level 4?

Deleveling’s revolutionaries would have us believe an educational emergency militates for expanded implementation of their preferred reform right away. But the only emergency is their fear that a political moment will pass them by. We should let it pass. There is time to learn what works and what doesn’t in our middle schools, and the Board of Education needs that time to fulfill its repeated promises to analyze objectively, not as advocates, the measurable outcomes of restructuring. The public won’t even have access to complete, disaggregated NJASK data until 2012. 

But the district’s leadership appears to be closing ranks, arming itself with selective information, and stalling on sharing unfiltered data with members of the public who’ve asked for it. It’s almost as though there’s something they’re afraid too many people might realize between now and the day they delevel the high school.