A right to data is meaningless without knowledge of what is available

We agree with the statement that “the right to data is meaningless without knowledge of what is available” and with the proposals to create a central, easy-to-use catalog or inventory of all available information. We think this should:

– Include information from what is available not only in central government data sets, but also in those of more fragmented organizations, such as local authorities or the police, to allow for comparisons.

– Be easy to search with common sense terms.

– Include notes about how the data is updated and how often users can expect it to be updated (ie, if it’s a one-time release, quarterly, including the next release date)

– Include links to other related data sets if they are part of a historical data series (or with other breakdowns, such as relevant regions or agencies)

– Presented in a user-friendly format, as well as raw format where possible, including accessibility star rating.

– Allow user feedback, rank the interest/value of datasets, and opportunities to publish relevant applications based on the data. Using these approaches can help make data inventories self-regulating and reduce the effort required to manage and maintain them.

The Government may also wish to highlight, point or even implement visualization tools to encourage less experienced users to access and manipulate the information.

At a high level, individuals (by which we mean in particular the group of specialized developers) will comply if the standards are sensible, the cost of compliance is low, and compliance itself does not cause further interoperability problems. A useful step would be to encourage consistent schemas for particular data sets (eg bus schedules) and ensure that they are interoperable with other related sets (eg train schedules). We recommend that clear lists of schemas with links to their definition be published on the same website as the catalogue. Also, it would be useful to have consistent master data across all relevant government datasets (for example, on hospital or station naming) as it will help with navigability, usability and interoperability. Another step to increase usability would be to ensure that key data sets are available (and easy to query programmatically) on government servers, as well as downloadable. This makes creating mobile phone apps, for example, much easier when the information is updated regularly and means individual users don’t have to download the entire data set. It is appreciated that the cost of maintaining such a service may require restrictions on the number of queries an individual service can perform; this could be fixed by using a private API key for each user of the service in a similar way to Google Maps. In addition, we suggest that the government adopt and communicate to employees clear information governance measures to achieve compliance and ensure usability and interoperability. Based on our experience working with public and private service organizations, we have developed information governance frameworks that can be applied to the US public services and open data agenda. The information governance framework examines measures to maintain the privacy, confidentiality, security, quality and integrity of data. Two of these areas are of particular relevance to achieving usability and interoperability:

1. Rigorous data hygiene standards must be adopted to improve data quality. Ensuring data quality is a major challenge, especially in complex environments with multiple IT systems, not all of which share common technical, data, communication, or terminology standards. The key to ensuring data quality in these environments is developing standardized interfaces and models that enable IT subsystems to share information effectively. Effective system architectures must include key components:

– Manual and automatic processes that detect and correct information errors efficiently and effectively. Emphasis should be placed on incentivizing and motivating public service professionals to understand the implications of poor data quality and to change behavior to improve data quality over time.

– Validation rules that verify that the data conforms to a set of specifications in terms of format, quality, completeness, precision, and structure.

– Use open standards for data recording and coding to promote a high level of data quality through similar data processing in multi-component systems.

2. Quality data must preserve its integrity when it is stored, transferred or retrieved. Unauthorized modification of data, low-quality source code, and non-interoperable subsystems undermine data integrity and thus the open data agenda. Effective information governance architectures to maintain data integrity must include:

– Processes for testing the source code to eliminate errors (which may result in data loss or corruption);

– Processes that identify and mitigate security risks;

– A governance function that works across silos to develop and enforce common standards, protocols, and processes to enable syntactic, semantic, and/or process interoperability;

– A standards-based system architecture conforms to open or common standards for messaging, infrastructure, communication, application, data, and clinical terminology.

Standards must be established to ensure that the data collected is consistent and easily comparable between agencies, public service providers and departments. It would also be worth considering the feasibility of synchronizing the dates on which the data is updated to ensure consistency in future comparisons.

Given today’s freedom of information requirements, public service providers are already balancing a commitment to openness with the need to respect privacy and security. We believe many of the same principles can be followed to ensure a commitment to open data.

There are three main areas where we would expect the government to routinely collect and publish data, particularly where the publication:

– Improves results and increases the productivity of utility providers through informed comparison;

– Supports the choice agenda, informing citizens about different providers and alternative services, which underpins the market;

– Makes accountability real for citizens and encourages greater engagement with public services and government.

We believe that the most important thing that government can do to stimulate the open data market is to make the data itself more consumable and accessible. Data is most useful to the citizen when it tells a story and is meaningful. By investing limited resources to move from “data” to “intelligence”, the government can lead by example and show the potential of data sets. It will also help inspire entrepreneurs to explore the potential uses of public data for business benefit, thereby fueling economic growth. It may also be worth going a step further and transforming some crucial data sets into bundled data “services” making them easily consumable. This will help establish a market and stimulate demand for more published data sets.