Let’s say, out of the blue, someone tells your child that she is one of the top 50 students in her class of several hundred students.  You know that your child is bright, inquisitive, and hard working, and has a lot of things going for her, but you also realize that she faces some serious challenges.  What would you say to her?  Would you tell her that her new ranking means she is doing an excellent job?  Or would say to her that a “top 50” rating might not be something to dwell on too much?

That’s how I felt when I saw that New Jersey Monthly recently ranked Columbia High School (CHS) as the 47^th best public high school in New Jersey, putting the school in the top 20% of 328 ranked public high schools.  According to New Jersey Monthly, CHS’s ranking has improved over time, from 89^th in 2008 and 75^th in 2010, to 47^th today.

That sounds great doesn’t it?  The School District certainly thinks so.  Superintendent Brian Osborne put out an official statement commenting on the new ranking:  “The entire community should be proud of CHS’s movement up the rankings and the accomplishment of our students that the rankings represent.”

But what do the rankings really represent?  I was curious so I decided to take a closer look.  New Jersey Monthly’s ranking methodology uses three categories of criteria to rank schools:  School Environment, Student Performance, and Outcomes. [1]

As soon I started comparing CHS to other schools using these criteria, I sensed that something was a bit off.  The first clue was the much lower rating given to Montclair High School (99^th), a school that is very similar to CHS in terms of student diversity, school culture, and community socioeconomics.

To evaluate the soundness of the New Jersey Monthly ranking, I decided to compare CHS to similarly ranked schools, such as Westfield High School (49^th).  Here are the ranking criteria for Westfield and CHS.

Westfield (Ranking 49, District Factor Group I)

• 457.5____Grade 12 Enrollment 
• 20.0_____Average Class Size 
• 11.7_____Student / Faculty Ratio 
• 63.4_____ % of Faculty with Masters or Doctorate 
• 1732_____Average combined SAT Scores 
• 46.2_____% Advanced Proficient HSPA Language 
• 54.8_____ % Advanced Proficient HSPA Math 
• 21_______Number of AP Tests Offered 
• 89.2_____% AP Tests Resulting in 3+ 
• 97.2_____Adjusted Cohort Grad Rate 

CHS (Ranking 47, District Factor Group I) 

• 406______Grade 12 Enrollment 
• 19.9_____Average Class Size 
• 9.9______Student / Faculty Ratio 
• 62.3_____% of Faculty with Masters or Doctorate 
• 1556____Average combined SAT Scores 
• 33.7_____% Advanced Proficient HSPA Language 
• 37_______% Advanced Proficient HSPA Math 
• 24_______Number of AP Tests Offered 
• 83.8_____% AP Tests Resulting in 3+ 
• 91.9_______Adjusted Cohort Grad Rate

Link to New Jersey Monthly rankings.

And below is a comparison of the various ranking criteria, using a + or – to indicate which high school has the advantage on that particular criteria.

• Average Class Size: +CHS by 0.1 (rough TIE)
• Student / Faculty Ratio: +CHS by 1.8
• % of Faculty with Masters or Doctorate: +Westfield by 1.1% (rough TIE)
• Average combined SAT Scores: +Westfield by 176
• % Advanced Proficient HSPA Language: +Westfield by 12.5%
• % Advanced Proficient HSPA Math: +Westfield by 17.8%
• Number of AP Tests Offered: +CHS by 3
• % AP Tests Resulting in 3+: +Westfield by 5.4%
• Adjusted Cohort Grad: +Westfield by 5.3%

As should be apparent, Westfield is “winning” in the majority of New Jersey Monthly’s criteria, and not just by small amounts.  For example, average SAT scores at Westfield are 176 points higher than at CHS.  CHS is better in only two categories:  Student / Faculty Ratio and Number of AP Tests.  And yet, the two schools have roughly the same ranking in the survey.  How is that possible?

I ran a rough simple regression to try to simulate the rating model that New Jersey Monthly uses.  Based on my calculations, I estimate that a one point difference in Student / Faculty Ratio translates into about 7 or 8 ranks.  In other words, the New Jersey Monthly rating model appears to place a great deal of weight on Student / Faculty Ratio.  But interestingly, in the case of CHS, that lower Student / Faculty Ratio did not actually mean smaller classes than Westfield, as both schools had roughly equivalent Average Class Size.  On the other category where CHS has an advantage – Number of AP Tests – a comparison doesn’t tell us much because there is often no straightforward answer to how many AP classes a school has.  In particular, schools sometimes combine related AP classes in a single section, which could prepare a student for multiple AP exams. [2]

So even if you believe that rankings like this matter (and I have some doubts), the New Jersey Monthly rankings seem questionable.  Of course, I don’t fault the School District for talking up the ranking.  As a marketing and promotion tool for CHS, the ranking is good for property values and helps the School District attract residents and educators.  At the same time, if any school official pointed to these rankings as indicating where the best teaching and programs are taking place, I would be quite skeptical, particularly because rankings like this tend to bounce up and down, year by year. [3]  If we fall twenty ranks in the next New Jersey Monthly survey, I don’t think that would be conclusive evidence that we are doing something wrong.

All of this is not to say that CHS is not a good high school.  In fact, I believe that there are many wonderful things happening at CHS with truly dedicated teachers committed to excellence.  One objective measure of that excellence is to look at the quality of individuals who have graduated from CHS, in both the past and the present.  A quick review of the Wikipedia page for CHS lists dozens of notable alumni like famed sex-researcher Alfred Kinsey (1912) to author Paul Auster (1965) to former Daily Show Executive David Javerbaum (1989) to musician Lauryn Hill (1993). Indeed, in 2007, Christian Sahner (2003) became the second Rhodes Scholar among CHS alumni. Throughout its history, CHS has produced an astounding number of high profile graduates.  As a point of comparison, the Wikipedia page for New Providence High School, the top ranked (1^st) New Jersey public high school according to New Jersey Monthly, lists only four notable alumni. [4]

Another way to understand the strength of CHS is to recognize that its student body is much more racially and economically diverse than other schools and so CHS faces unique challenges.  Yet, in spite of those challenges (or perhaps because of them), the school produces some very good student outcomes.  Below is a comparison of the scores of White students on the 2011 New Jersey High School Proficiency Assessment (HSPA) at CHS, Millburn High School, Westfield High School, and New Providence High School.

HSPA LANGUAGE ARTS

• CHS (47^th NJM Rank)
  White students - Mean Scale Scores: 249.6
  - Percentage Scoring Adv. Prof.: 57.7%
• MILLBURN (6^th)
  White students - Mean Scale Scores: 247.9
  - Percentage Scoring Adv. Prof.: 49.8%
• WESTFIELD (49^th)
  White students - Mean Scale Scores: 244.7
  - Percentage Scoring Adv. Prof.: 46.5%
• NEW PROVIDENCE (1^st)
  White students - Mean Scale Scores: 244.2
  - Percentage Scoring Adv. Prof.: 42%

HSPA MATH

• CHS 
  White students - Mean Scale Scores: 250.4 
  - Percentage Scoring Adv. Prof.: 61.4%
• MILLBURN
  White students - Mean Scale Scores: 249.7
  - Percentage Scoring Adv. Prof.: 66.9%
• WESTFIELD
  White students - Mean Scale Scores: 246.5
  - Percentage Scoring Adv. Prof.: 55.9%
• NEW PROVIDENCE
  White students - Mean Scale Scores: 247.7
  - Percentage Scoring Adv. Prof.: 62.2%

Link to New Jersey HSPA Data.

The above shows that White students at CHS perform as well or better on these State assessments than White students at the other top-ranked high schools.  A similar analysis shows that Black students at CHS perform as well or better than Black students at high schools with significant Black populations (such as Montclair High School). [5]  Now, I believe that we should not use diversity as an excuse.  I do not like breaking down people into racial categories, and I think our School District does too much of this, but in order to make a fair comparison, you cannot deny the demographic reality.  When you take into account our diversity, CHS does very well. [6]

The bottom line is that CHS is an excellent school by a number of objective measures.  However, New Jersey Monthly’s rankings are problematic and should not be used as a benchmark for success.  If we want a real evaluation of how we are doing, we need to dig a little deeper.  And, in a future article, I will dig deeper and discuss some of the challenges that CHS faces, and how the School District should move to address them.

_________________

NOTES:

[1] New Jersey Monthly’s methodology relies on three categories of indicators to rank schools:  School Environment:  The sum of the standardized rank scores for average class size; student/faculty ratio; percentage of faculty with advanced degrees, and number of AP tests offered.  Student Performance:  The sum of the standardized rank scores for average combined SAT score; percentage of students showing advanced proficiency on HSPA, and students scoring a 3 or higher on AP tests as a percentage of all juniors and seniors.  Student Outcomes:  A single score based on a new graduation-rate calculation (four-year adjusted cohort graduation rate) introduced by New Jersey in 2011, as mandated by the federal government.

[2] For example, in previous years, CHS combined AP Comparative Government and AP US Government into a single class.  In Westfield's case, the figure of 21 AP Tests is also probably an overstatement of actual classes, since according to NJDOE statistics, three of Westfield’s AP Tests (AP Environmental Science, AP German, AP History of Art, and AP Macroeconomics) had no students in the accompanying class and only a handful of students taking the tests.

[3] In response to New Jersey Monthly’s new ranking of CHS, Board of Education President Beth Daugherty stated that “CHS and district leaders have had a strategic commitment to excellence and equity over the past several years that is moving Columbia High School forward.”  CHS Principal Dr. Lovie Lilly chimed in that “these rankings are further evidence that we are moving in the right direction.”  I have to believe that these statements are simply part of the marketing and promotion of the School District, and do not reflect the Board of Education and Administration’s true beliefs about what the rankings actually mean. 

[4] New Providence High School was founded many years after CHS, so a comparison of notable alumni might not be the fairest measure.  Nevertheless, you would expect that the top-ranked public high school in New Jersey would have had many recent notable graduates.

[5] The number of Black students at Millburn, Westfield, and New Providence are relatively small and make statistical comparisons to CHS less reliable.

[6] The New Jersey Monthly ranking methodology does not mention diversity or demographics of a student body as part of their criteria for evaluating schools.