Public Data and Public Data Crunching

Things are usually pretty quiet in Keene, where I live, but over the last year or so we've seen some violent assaults, and a string of burglaries, and people have been wondering: Is this a pattern? Are we having a crime wave?

When the Keene Sentinel put the question to readers on its website, many seemed to think so. "Keene has gone downhill," one person wrote.
"Once a peaceful, quaint city that was safe, it is no more."

Of course if you really want to know how safe we are, you need to look at some data. A decade ago you couldn't find much of that on the web, but that's changing. From an FBI web site, you can download national, state and local data about crime. From that I learned that Keene did have a crime wave in the early '90's. Anyone could have discovered that useful bit of historical context.

Of course, it isn't enough just to have the data. You also want to be able to share your data and engage others in a conversation about it. Until very recently, there weren't easy and effective online tools for doing that, but in the last year or so, some interesting ones have started to come on line. At websites like Many Eyes, Dabble DB, and Swivel, you can upload a spreadsheet, chart data in various ways, and then invite other folks to discuss your interpretations -- or even make their own.

So what is going on in Keene? While there's been a slight uptick since 2006, it's nothing compared to what happened in the early '90's. Now I can't be completely sure of that because, frankly, data collection is a messy business. Like all organizations, government agencies use patchwork quilts of tools to gather and report data. Sometimes the numbers just don't add up. For example, I was looking at the stats on violent crime in New Hampshire between 2000 and 2004 and I found that some towns, including Durham, Salem, and Nashua, didn't report any crime data for those years.

Anyway, here's the deal. Governments are becoming more transparent and they're publishing more of our data online. At the same time, we citizens can use an emerging breed of online tools to collaboratively make sense of that information and help raise the level of public debate.

See a collection of related posts on this topic here.