New Hampshire’s scores on the latest federally mandated standardized test – the Smarter Balanced – were released Thursday.
The headline: Fewer than half of the state’s students were judged to be meeting grade level benchmarks in math, though they are doing somewhat better in English.
But there was another piece of news buried by the announcement. It wasn’t part of the press release, but for this new test the state Department of Education has made a decision that will make it nearly impossible to use the data to answer really important questions about how well Granite State students are doing.
Say you’d like to check how your school did; that’s easy enough. You can navigate your way over to the Department of Education’s website, look up your school profile and -- Presto! There it is.
Now, say you’d like to take the next obvious step and compare your school to the state average score. That’s also pretty easy: the state averages are also prominently posted. It’s also fairly easy to get reports of how a student subgroup – say, white students or English Language Learners – is doing in an individual school.
But what if you wanted to go deeper? What if you wanted to compare poor students from schools in Coos County to those in Rockingham County.
Now you’re getting into tricky territory.
“Essentially the data in their current form, by putting them in the school profiles are virtually useless,” says Joe Miller, an education consultant from the Upper Valley who has written a letter to the governor lamenting the DOE’s decision to sit on that more detailed test info.
He says in past years, more detailed scoring data was available for schools, researchers, and the media. That’s not the case this year.
“I would say that there aren’t many reasons not to do it, and so this is possibly to discourage a close look and a comparison of schools,” Miller speculates. He says this is part of a pattern of obfuscation he has observed at the department when it comes to accountability measures.
Contractual Problem
The state, for its part, says this is not deliberate.
“In the past, our previous vendor actually supplied a file that way so it was part of the contract, and our current vendor doesn’t supply any kind of information like that,” explains Scott Mantie, who has been helping the state to manage the data.
He says this is simply a contractual problem. Previously, the company was a New Hampshire-based outfit called Measured Progress. Now, the data comes a vendor called AIR - the American Institutes for Research. The state says the contract with Smarter Balanced is “very extensive” and simply didn’t contain a provision for this kind of file.
Paul Leather, the deputy commissioner at the DOE, explains that schools can use the department’s school profile comparison system to see how they stack up to other schools.
“Essentially, we feel that you can get the kinds of information that you are looking for by using the profile, so that’s what we’re encouraging you to do,” he explained.
As of press time, it was possible to look up a one school’s profile at a time, but any attempt to use the comparison tool yielded an error message.
“You’re Allowed to Write the Story…”
Though the department does have this data, but it’s limiting how much is available on its website.
“There’s a database that connects up into the webpage and populates it up into the webpage that way,” explains Mantie.
Why not make that public?
“Well it’s a large… it’s a database that is completely relational, that has privacy information,” Mantie begins, explaining the data is tangled up with other student level information that's protected by state privacy laws.
“This would require a substantial effort by programmers to pull this together,” he concludes.
So even though this kind of data set was available for ten years under an older standardized test, with the rollout of this new -- and controversial -- test, it’s suddenly not. Both Vermont and Maine – who also use Smarter Balanced as their statewide test – make this data available on their department of education websites.
Morgan Polikoff, an assistant professor at the University of Southern California Rossier who works with this kind of data set all the time, says he thinks the spreadsheet in question probably already exists, and is being used to generate school level reports.
“But even assuming that they didn’t exist, if you’ve already got the data calculated it should be quite easy to put together a report.”
Palikoff says strictly from a research perspective, he can’t think of any reason why the state wouldn’t provide this data.
But consultant Joe Miller has a theory.
“By controlling the information you’re allowed to write the story of what’s happening with education in New Hampshire, and I think by being a bit more transparent and releasing the data they’d allow the public to interpret the data themselves,” he says.
Today, New Hampshire Public Radio filed a Right to Know Request, asking that the state do just that.
**Corrections: A previous version of this story misspelled Scott Mantie's name, and misstated the company the state is using as a test vendor. It is Mantie, not Manti and the state is using the American Institutes for Research as their vendor and don't get the data straight from the Smarter Balanced Consortium.**
