Written yesterday at the EA Conference held in Washington, DC.
This morning's session on data.gov was really nothing short of inspiring. There has been a sea change in how government data is made public. As little as a year ago, even government RSS feeds were presented in such a way as to be barely re-usable, as if their agencies were providing open data under protest, and doing as much as possible to keep their data secret.
Contrast that to the accomplishments of data.gov today, with their tens of thousands of data sources, RSS feeds that really expose data, application contests to do interesting things with public data.
I asked the data.gov panel at the Government EA conference this week in the Ronald Reagan building what had changed. This seems like a difference of work culture in the agencies. What was the cause of that?
I got insightful answers from all the panelists. I don't want to put words into their mouths, so I won't attribute any particular answer to any of them, but the panelists were Sonny Bhagowalia (DOI), Jerry Johnston (EPA), Marion Royal (GSA) and Martha Dorris (GSA).
There are a few forces that are coming together to cause this change. First, there are people in the agencies who have always believed in open data, and wanted to share it, but have not had a charter to do so. They have effectively done it in their spare time, just waiting for a chance.
The efforts that they have managed to make have been oriented toward very specific tasks; they made data available in a way that they thought some particular consumer wanted it. This would allow them to justify the effort of publishing the data. But data presented for a single consumer doesn't feel like 'open' data to the rest of us; it can even feel as if the data is being kept intentionally secret. Early feedback (early? As recently as June the whole effort was called a "significant failure" on this point alone) to data.gov told the providers that there is an audience for 'raw' open data. So they have started to do both.
Another force is hard times. This country is in the midst of a number of crises, and the government is involved to a great extent in the problems and any solutions. Government data is more important than ever. And the agencies need to harness the ingenuity of the masses to work through it, adding another incentive.
This situation is like a powder keg ready to go off. We have people in the agencies who want to share data, who want to stimulate the clever folks at MIT or Stanford or in their garages to solve problems using government data, and who want to get around requirements for particular audiences for their published data. To this mix, you add a spark: in February, President Obama signed the memorandum about Transparency and Open Government.
Critics might cry that this is too little, too late. But the gains that data.gov has made in the past few months show a real change in attitude; a far cry from what we had before.