OPEN DATA PORTALS SHOULD BE API [FIRST]

-

As you can see from the figure, September and November are not far below October. Some readers may wonder why there is a surge in API calls starting in May of 2014. May through October was spent building open source service architectures on Red Hat JBOSS Switch Yard that could mine and automatically append data sets within the Open Raleigh Portal.


Open Raleigh uses a responsive web design that is friendly to most handheld devices but the API needs a little help to push data into the portal. The portal itself releases every data set as an API endpoint. This API is a READ only API. Writing some code we can have the Socrata portal allow us to append data sets.


Socrata is not alone in the Web/Mobile [First] category. ESRI, CKAN and to some extent, Junar are architected on the same principals. This is not a direct criticism or endorsement of any particular platform.


THE CONSEQUENCES OF GETTING IT WRONG

Discussing multi-nodal approaches and espousing an API [First] strategy may seem esoteric until one looks at anecdotal issues with some recent portal launches.


Minneapolis recently launched its open data portal to scathing reviews (http://blogs.citypages.com/blotter/2014/12/minneapolis_new_open_data_portal_is_a_disaster.php).


Most of the criticisms were centered on the performance of the site, latency, non-responsive design and crashing web pages. Note that these were all complaints from citizens trying to use the portal through different browsers. The city blamed ESRI but latency and poorly designed pages that do not validate are not inherent in the platform. ESRI is not an API [First] product. The city said it should have gone with Socrata. Given the city’s ability to manage a roll out it seems clear that the lack of a multi-nodal standards based approach was a significant but not a single-cause reason for the beta failure.


CKAN also recent announced the launch of a new comprehensive open data portal on the Ebola crisis through a tweet. Here is the URL:


https://data.hdx.rwlabs.org/ebola

This is not responsive design. When I look at it on my mobile I see a tiny version of the full site and not anyway to meaningfully consume or re-use the data unless I switch to a larger interface.


Now let’s think of the consequences of that:


Who is the data for? If it is for field workers this is a huge fail. The most common field device is tablets and mobile. Not having some kind of app consuming an API would be an obstacle toward data re-use.

How do I consume and re-use the data? Following the rabbit hole of links I can get to geo-data about the crises data. Most of the catalog lists are CSV data sets along side PDFs giving the context of the data. This is good in that I have metadata. Bad in that I cannot query an obvious API point to merge this sites data with other data for my own analysis.

CONCLUSIONS

This is only the tip of the iceberg. Aside from the technical issues around not using an API [First] strategy we have policy issues around PII (In the case of Minneapolis) and UX issues in the CKAN instance.


So, I conclude the following by comparing human and API consumed data:


Data consumed by humans have lower re-use value in that they are not being redistributed

Data that is served on a Web/Mobile [First] platform needs more work to reuse data than platforms that are API [First]

Original article: https://www.linkedin.com/pulse/open-data-portals-should-api-jason-hare?trk=mp-reader-card

Comments

Post a Comment

Popular posts from this blog

Podcast: Open Data Discussions with Anthony Fung

-
Image
A look back at Virginia’s 4 years of data innovation with Anthony Fung, Deputy Secretary of Technology for the Commonwealth of Virginia  Welcome to the OpenDataSoft Leadership Podcast Series, “Open Data Discussions”. Each month, Jason Hare, our Open Data Evangelist, features a different open data program around the country to discuss what has made it successful. These examples will provide insights and strategies that you can implement in your own city. We had the pleasure of welcoming Anthony Fung, Deputy Secretary of Technology for the Commonwealth of Virginia for the fifth podcast in our series. Anthony discussed various subjects, including: What role does open data play in policymaking at the state level in Virginia? Does his department work on ensuring open data quality? What are some of the steps he makes in ensuring sound data governance? How did Virginia come up with this technology and engagement solutions?  How, for example, did yo
Read more

Open Data Licensing

-
Image
Everything you wanted to know but you were afraid to ask. Making data open and available is not just about choosing a format or having an API. A poor choice in your license can prevent people from using them. A traditional open license reassures potential users that their work on the data will be useful in the long run. So, what license should you pick for your data to be broadly used? There are a few things to consider. Choosing a license for your open data can usually be one of the biggest challenges for an open data project. There are plenty of licenses available for your data. Choosing one mostly depends on the usage and reach you want to give to your project. CHOOSE A LICENSE: BASED ON NATIONAL, INTERNATIONAL AND LOCAL STATUTES IN ADDITION TO REUSE It should be noted that different parts of the world use data licensing terms differently. In the US, works in the “public domain” are works are not covered by intellectual property rights, such as copyright. The copyright mig
Read more

AN OPEN DATA POLICY FOR NORTH CAROLINA: COMING SOON?

-
Image
All parts of North Carolina will benefit from open data legislation. FOR THE BETTER PART OF TWO DECADES, I’VE WORKED TO OPEN UP NORTH CAROLINA’S DATA. I’ve been a public servant at Durham Public Schools and the Department of Energy. I have been an onsite consultant for the Cities of Raleigh, Durham as well as the Towns of Cary and Chapel Hill. I’ve been on teams serving state agencies and local governments. I’ve witnessed dozens of open data projects both locally and abroad. All of these projects share crucial steps and goals in common. First, by transforming public documents from disconnected PDFs into machine-readable data, by applying the right open formats, we can liberate public information from the repositories where it used to require manual review. Insights become instant. Second, by publishing machine-readable data compilations online, for everyone to use, we can crowdsource those insights. Citizens and companies and the media can scrutinize. Even better, when a government age
Read more