On measuring school quality, ‘Education Week’ misses the mark

BY IAN KINGSBURY

Education Week released its A-F grades for states on school quality last week. New York earned a B-, appreciably better than the nationwide C average. A closer look at scoring methodology, however, shows grade inflation at work.

Education Week raised New York’s middling academic achievement grade by giving it high marks for spending—one of three indexes they use to grade states. Education Week calls the index ”Finance,” which measures expenditures and how equitably funding varies across districts. This includes, for example, to what degree district revenue correlates with real estate value.

New York received a score of 89.3 (or B+) for finance, the third highest in Education Week’s rankings, trailing only Wyoming and New Jersey. That ranking is consistent with New York’s rank as the highest spending state in the nation.

When it comes to measuring school quality, however, spending is a dubious measure. Today’s education research field can be traced to the observation that educational inputs are not especially predictive of outputs.

A more sensible question asks how well money is spent. New York receives a failing grade when comparing spending to results. Indeed, mediocre results despite highest in the world educational expenditures ought to be met with scorn, not praise.

New York gets a C in “K-12 achievement.” The index measures standardized testing outcomes, high school graduation rates, and the percentage of students who score 3 or higher on advanced placement exams.

New York scores 72.6, a tick below the national average of 72.8. Those marks are consistent with state results on NAEP, a national assessment which facilitates comparisons of learning outcomes across states. New York trends in NAEP indicate that achievement is no better or worse than it was two decades ago.

National scores, however, modestly improved, so New York’s performance declined relative to the national average. Whereas New York previously scored above the national average on NAEP, achievement now trails the national average.

Notably, achievement outcomes reinforce that New York’s higher spending has not been a proxy for school quality. If that were true, New York schools would have improved markedly over the past 15 years. Instead, achievement stagnated.

New York’s low achievement grade is also bolstered by a B- in the third index in Education Week’s ranking system—“Chance for success.” That index measures demographic characteristics predictive of educational achievement and attainment, and includes measures such as family income and the percent of adults who are steadily employed.

New York earns an 82.2, which compares favorably to the national score of 79.2. However, including a chance for success index as a school quality measure is questionable. A great deal of school achievement is explained by demographic and socioeconomic characteristics.

Researchers expend enormous efforts to isolate student socioeconomic status from school quality when measuring achievement. Otherwise, most schools with affluent students would be labeled “good schools” and most schools with underprivileged students would be labeled “bad.” A more sensible question asks how school achievement deviates from schools with similar student demographics.

It may be that measures such as wealth and employment can be traced back to school quality. But evaluators shouldn’t assume that all New York residents were educated in New York. And such metrics don’t factor in that their education may have occurred decades ago. The “chance for success” index allows New York schools to get a lift on their Education Week rankings simply because the state is wealthy, an arrangement that is not easily reconciled with prevailing research sensibilities.

All told, Education Week’s rankings do not measure what counts. New York’s substandard achievement coupled with highest-in-the-nation spending and above-average wealth means that when it comes to school quality, New York fails to pass the mark.

Dr. Ian Kingsbury is the Empire Center’s fellow for education policy.