SFWeekly critique, part 2

Small schools are a fine thing for many students who struggle with the bustle of a larger setting. They’re not for every kid, especially high school students who seek the programs that larger schools can offer, such as a wide choice of electives, foreign languages, arts, athletics and clubs.

Small schools are also the trend of the moment among a segment of the "school reform" movement. The May 3 San Francisco Weekly published a badly flawed article about the San Francisco Unified School District that — while nominally disclaiming that small schools are a panacea — portrayed them as a panacea.

The article was marked by inaccuracies and firmly stated claims about data that is actually misinterpreted, unattributed, confusingly spun or unverifiable. Its story line about an uncaring bureaucracy spurning a shining opportunity does not hold up to scrutiny.

The article, "A Study in Size" by Ryan Blitstein, created an inaccurate impression that achievement at SFUSD small schools is surpassing that at larger schools with comparable populations. Yes, some students find the small-school setting ideal, but the notion that small schools are resoundingly successful and are turning away students — and that SFUSD is missing an opportunity in failing to create many more — is not borne out by fact.

Here are some points about the San Francisco Weekly article, which showcases the June Jordan Small School for Equity in the Excelsior District.

continue reading...

• The article misleadingly indicates that June Jordan students are outperforming their peers districtwide on the California High School Exit Exam (CAHSEE). But the most recent publicly available data on the CAHSEE for 10th-graders (the only valid comparison, as June Jordan is too new to have graduated a class of 12th-graders yet) shows a pass rate of 66 percent in math compared with a districtwide pass rate of 72 percent. June Jordan’s English Language Arts (ELA) pass rate, at 71 percent, is slightly higher than the districtwide average of 66 percent, but June Jordan has significantly fewer English-language learners than the districtwide average for high schools — 11 percent compared with 18 percent districtwide (a factor strongly impacting ELA test scores). The article cited a claimed, unverifiable pass rate for June Jordan 11th-graders this year of 93 percent, but that figure is not yet publicly available.
• The article creates an impression that small schools are achieving "impressive performance" compared with other schools. But the facts don’t support that claim.
  • The article used students’ grade point averages to back up that statement about June Jordan, though grade point averages are never viewed as a reliable measure. That’s because grades are subjective, and standards of rigor can vary widely from school to school. It’s unknowable whether high grades at a particular school reflect improved achievement or grade inflation.
  • Standardized test scores for Leadership Charter High School, one of the small schools cited as a success, have dropped steadily for several years. Nearby Balboa High School, a traditional high school, is cited in the article as the low performer to Leadership’s success. Yet Balboa’s test scores have been steadily rising as Leadership’s have fallen. In 2001, Leadership outperformed Balboa by 271 points on the state’s API (of a total 800 points), while by 2005, Balboa's steadily improving test scores had narrowed the gap to 69 points.
  • John O’Connell High School, also cited as an unsuccessful traditional model, has posted an API increase of 24 percent over four years. June Jordan’s API has been reported for only two years, but dropped slightly from the first to the second year.
  • Aim High, a small middle school also portrayed as a success in the article, failed to achieve almost all of its mandated Adequate Yearly Progress targets under the No Child Left Behind federal requirements. And the school, which is in its third year of operation, improved its API score by only one point (on an 800-point scale) from its first year of operation to its second year.
• As previously noted, the article indicated that "record numbers" of applicants are trying to get into the small schools mentioned. But actually, both June Jordan and Aim High had considerably fewer first-choice applicants than openings. Also as noted, the article claimed that no other small schools had opened in the district, overlooking the Academy of Arts and Sciences on the School of the Arts campus. In fact, the Academy of Arts & Sciences was the only small school that attracted more first-choice applicants than openings (105 first-choice applicants and 606 total applicants for 80 openings for fall 2006).
• The article focused on one student, showing him as disengaged at O’Connell High before successfully transferring to June Jordan. Of course the student’s and his family’s satisfaction with June Jordan is a valid and significant point to report. However, the actual measures of success as described in the article were an apparent slight increase in grades — from "B’s and C’s" at O’Connell to B’s at June Jordan — and improved attendance. But the student lives across the street from June Jordan, as opposed to 4 miles from O’Connell — a huge confounding factor strongly likely to improve attendance.
• The article gave a figure that perhaps inadvertently revealed significant student turnover at June Jordan, with no further comment. The fact, mentioned in the article, that 20 10th-graders transferred into June Jordan, indicates that 20 students left June Jordan, in a school with about 100 students per grade (publicly available figures show that the student population remained constant). A separate indicator, the California Basic Educational Data System, shows that 39 11th-graders arrived at June Jordan partway into the 2004-05 school year. That amounts to alarmingly high student turnover of 20 to nearly 40 percent in those grades.
• The article noted that the Bill and Melinda Gates Foundation slashed its grant funding for small schools in SFUSD, implying that this was due to flaws in SFUSD’s implementation. The article didn’t mention that the Gates Foundation has done the same thing in school districts around the country as results from investments in small schools fell short of expectations. "In our early grant making, small became the goal. To the extent it became the main focus, that wasn't productive," Gates Foundation spokesman Tom Vander Ark told the Seattle Weekly in July 2005.
• The article portrayed Oakland Unified School District flatteringly, as far more welcoming to small schools. But troubled Oakland Unified compares unfavorably to SFUSD on every possible gauge, making it a startling and unlikely choice of role model. Oakland test scores are far lower than SFUSD’s, and the Oakland district was taken over by the state a few years ago after its financial management imploded.
• The article noted almost in passing that Aim High, a small middle school, is in management turmoil and its future is in question, and blamed "lack of support from the district." But the article didn’t mention the crucial fact that Aim High has not met its mandatory Adequate Yearly Progress targets. In assigning blame for the school’s problems to conflict with SFUSD, the article vaguely attributed the charge to "several people who have spoken to Aim High board members and executives" — indicating that the reporter had not spoken with Aim High officials himself but was merely repeating gossip, sourcing that falls far short of professional journalistic standards
• Aside from the measures already mentioned — such as the unverifiable claim about CAHSEE success and the methodologically irrelevant use of GPAs — the article relies entirely on the Similar Schools rank from California’s Academic Performance Index system. The API uses both a statewide absolute rank and the Similar Schools score, which attempts to control for demographics. The statewide rank is considered the essential one, with the validity of the Similar Schools rank continually debated. As one school superintendent (John Rieckewald, Fremont Unified) put it, the Similar Schools rank "is imperfect at best, because schools aren’t always compared to those that have the same demographic characteristics, the comparison schools can change yearly and the characteristics seem to be weighted differently. … Although interesting, the comparison rank is not as important to us as the API score itself." Statewide ranks dropped from 2004 to 2005 for both Aim High (from 3 to 2 on a 1-10 scale) and June Jordan (from 5 to 4).

We do not mean to disparage either June Jordan or Aim High, and we recognize that many students do well in a smaller, more personalized environment. But the San Francisco Weekly article is fatally flawed by the high number of errors, omissions and unsupported claims. Complete and accurate information refutes the implication that those small schools are resoundingly successful options in a world of struggling traditional schools, that they are turning away eager applicants, and that SFUSD is blindly ignoring their success by refusing to create more schools like them.

— Nestwife and Caroline