Colorado University professor of education policy and Education News Colorado contributor Kevin Welner Tuesday examined a report put out last month by Colorado Succeeds, a group of business leaders working for school reform. The report, Proving the Possible: A case study of Florida’s K-12 education reforms and lessons for Colorado (pdf), drew favorable coverage from the Denver Post when it came out. The Post, turns out, didn’t look at where the authors of the report were getting their information. Welner did.
Like reports put out by free-market school reform groups across the country, the Colorado Succeeds report was part of a marketing campaign pushing the Florida school reform laws and that leaned almost solely on a right-wing Heritage Foundation study with serious methodology problems outlined at length by Welner colleague and Columbia University Prof. Madhabi Chatterji. Welner calls the Colorado Succeeds study by extension seriously flawed and unable to weather scrutiny.
From a press release on Chatterji’s review of the Heritage Foundation research:
By analogy, consider growth in height instead of growth in test scores. If two states wanted to measure the average height of their fourth graders, but one state (Florida) first identified the shortest 20% of third graders and held them back to grow an additional year before measurement, the study’s results would not be useful.
That, in brief, is the key problem that professor Chatterji identifies with the Heritage report. Florida’s retention policy, instituted in 2002, focuses on third graders, who are held back when their reading scores are low. The Heritage report focuses on NAEP fourth grade reading scores. Low scoring readers—mostly black and Hispanic—were screened out of grade 4 tests, which resulted in inflated and erroneous fourth grade scores. “Chatterji’s review explains very clearly why the simplistic comparison of fourth graders before and after Florida’s grade retention policy is a predictable and worthless exercise,” says Kevin Welner, professor of education at the University of Colorado and the director of NEPC.
Even more damning, perhaps, Welner notes that at least many of the other conservative groups and outlets that brought out reports or articles based on the Heritage research– including the Hoover Institution, the Pacific Research Institute, National Review Online and Fox News– listed Dr. Matthew Ladner, vice president of research at Arizona’s free-market Goldwater Institute, as an author or co-author. Colorado Succeeds did not list Ladner as an author in its report. Yet it was Ladner who was pushing the Heritage research and, in effect, running the national marketing campaign for the Florida reforms.