AN OPEN DATA POLICY FOR NORTH CAROLINA: COMING SOON?

All parts of North Carolina will benefit from open data legislation.

FOR THE BETTER PART OF TWO DECADES, I’VE WORKED TO OPEN UP NORTH CAROLINA’S DATA.

I’ve been a public servant at Durham Public Schools and the Department of Energy. I have been an onsite consultant for the Cities of Raleigh, Durham as well as the Towns of Cary and Chapel Hill. I’ve been on teams serving state agencies and local governments. I’ve witnessed dozens of open data projects both locally and abroad.

All of these projects share crucial steps and goals in common. First, by transforming public documents from disconnected PDFs into machine-readable data, by applying the right open formats, we can liberate public information from the repositories where it used to require manual review. Insights become instant.

Second, by publishing machine-readable data compilations online, for everyone to use, we can crowdsource those insights. Citizens and companies and the media can scrutinize. Even better, when a government agency publishes open data, that agency’s own staff oftentimes become its most enthusiastic users.

Open data creates enhanced access and evidence-based governance. When insights leap out from formerly-inaccessible repositories of public information, policymakers can act on them. (Or, if the policymakers do not act, the voters can.)

Successful open data projects both realize the goal of a data-driven government in the short term and lay a foundation for greater transformation in the long term. In the short term, just using data sets and technologies that are already deployed in our government, citizens and public servants can scrutinize spending, uncover insider deals, and uncover patterns.

Over the long term, more comprehensive data and better technologies are going to allow open data to empower all sorts of transformations: automatic compliance with laws, autonomous transportation. Success depends on one thing: practical use! Open data projects work best when they meet existing demand, whether inside or outside government and also anticipate future demand.

There are also plenty of unsuccessful open data projects. These are the ones where governments spend a lot of money to set up an open data platform and curate some huge, one-off data sets for publication--but nobody uses them.

We need more successful open data projects, and fewer flops. To maximize the lessons of hindsight, we need some coordination amongst our state’s 179 distinct agencies and hundreds of local governments. To ease the pain of sharing data across different organizations, we need better data standards. And no open data project should proceed without consideration of use.

I’ve become convinced our state needs an open data policy--a framework to make sure that agencies coordinate their efforts, new projects embrace the same standards as old ones, and data users are listened to. So, I’ve decided to devote some of my time and money to change this.

We have partnered with Hudson Hollister, founder of the Data Coalition, to gather input from the users of open data--citizens, activists, practitioners, and the public servants who use open data within their own agencies--on what an open data policy for North Carolina might look like.

This week, Hudson and I traveled across the Research Triangle to have some initial meetings with some leading open data users, starting with leaders at the Department of Information Technology, the John Locke Foundation, the City of Burlington, Elon University, and the North Carolina Judicial Branch. We’ve learned a great deal already:

Our state’s agencies--some, but not all--are moving from a data ownership model to a data stewardship model.

There’s no way to organize and categorize all the state’s data from the top down. Instead, we need to empower users in specific, value-driven areas, like health care, criminal justice, and antifraud, to get the data they need and use it, cheaply.

Any push to transform a particular value area will take time and money. The state legislature recently funded a statewide Enterprise Resource Planning (system)--a system whose need has been recognized for a decade.

Local governments are enthusiastic users of state data sets--but especially when they have some notice and some say in how those data sets are structured. Open data and public records requests have a symbiotic relationship: the more an agency publishes the former, the less burdensome it will find the latter.

My goal is to reflect these insights in a responsible, lightweight yet proactive open data policy for our state. We’re only just getting started. There are many conversations--and, we hope, public convenings too--ahead.

An open data policy won’t privilege me. In fact, it will do the opposite: create an environment in which all tech firms and companies can compete to offer the best solutions. For now, I invite you to submit your thoughts on what an open data policy for our state should look like--and what bad ideas it ought to avoid.