Open data on its way

Core government agencies due to report on March 12

The government’s open data initiative is coming to a head, with core government agencies due to report back on March 12 on a request to list datasets in their possession which are of potentially high value for reuse and are not personal, confidential or classified.

The intervention of a general election has delayed the process somewhat, says Keitha Booth, leader of the Open Government Data and Information programme. Each agency has identified a “data champion” to encourage the process of discovering appropriate data and information. Agencies have been asked to focus particularly on data for which people have expressed a need.

All core government departments have been “directed” to develop a plan for actively releasing public data and to report back against that plan. State agencies are “encouraged” to do so; other central and local government organisations are “invited” to participate.

But the move to openness is not without nervousness.

“If you think you’ve got challenges now with controlling your data, who’s using it and how they’re using it, your challenges are about to get much greater if you’re in a government organisation,” Clare Somerville of business intelligence and data warehousing consultancy Knoware told an audience including a number of public-sector data managers at the annual SAS users’ of NZ (SUNZ) conference earlier this month.

Government agencies have been asked to identify what data they have released to date and what plans they have to release more, Booth says. A number of agencies have already released datasets via the www.data.govt.nz website and the public can lodge requests there for datasets of particular interest.

Data for release should, of course, be machine-readable, Somerville says. “It’s no good putting up a few [Microsoft] Word documents or PDFs to save a bit of printing or mail time. That won’t be much use to people who want to do something useful with [the data].”

Booth also says agencies should think beyond just the data that is easy to release.

Agencies have also been asked whether they have released or intend to release data in accordance with the NZGOAL framework, which details how the data can be reused, says Booth.

Government inclines towards liberal terms of licensing such as forms Creative Commons licences where possible.

“We could get a bit more adventurous and add a bit of metadata,” Somerville told the SAS conference. “We could get into sorting and visualisation. Or we could allow programmatic access to detailed data and [the creation of] large merged datasets.”

Data should be made available at “the highest possible level of granularity [detail],” says Somerville. “It won’t be enough to release a summary spreadsheet. We’re talking granular data, with metadata. You have to ask how many government departments will really be in a position to do this; do you know exactly what information you’ve got?” she challenged her audience.

“Do you have a management plan for it ? Do you know how you’ll dispose of it and on what date? Do you know its value? There’s a lot of stuff you need to know about good management of data before you release it into the wild.”

The focus, signalled in the Cabinet paper on Open Government, issued last August, is on data that will have economic, environmental or social value, relevance to transparency and democracy and/or value in increasing efficiency. The present government is likely to place the highest value on economic benefit, Somerville says; “the idea is that it will help the economy to grow, which will bring in more tax.”

As well as identifying data and information for new release and that which is released in the normal course of business, agencies are being asked what data they have avoided releasing because of “insurmountable barriers”. Both Somerville and Booth says identification of such supposed barriers will be one of the most interesting parts of the feedback.