Post: 31 October 2015
In the past year Environment Agency has made two significant commitments on open data:
First, that if EA publishes any new datasets for external re-use it will do so under an open licence. (Subject to the usual caveats about personal data, national security, third-party IPR, etc.)
Second, that EA will transition all of its currently charged-for data to open licences, on a proposed timescale that runs to April 2018.
That is a longer timescale than I would like. But EA at least has a coherent programme for release of nationally important open data. That puts it ahead of most other data-rich public authorities and assures EA of a lead role in the wider #OpenDefra initiative launched this summer by Environment Secretary Liz Truss.
Recently EA has given us major releases of flood data and LiDAR data, and there are now more than 100 other open datasets available for download from EA’s DataShare site.
However according to a new inventory Environment Agency holds over 1,300 identified datasets. At the current pace it will take EA many years to work through that list and release everything that is eligible for open licensing. EA also has a wealth of less structured environmental information, much of which is of potential interest to the public.
What’s missing from EA’s open data programme is a credible process for responding in a timely manner to user-generated requests for open re-use of specific environmental data or information.
EA’s open data programme is mainly focused on high-profile datasets. EA does consult quite widely to identify sector needs and prioritise release of the data with the most impact. The EA Data Advisory Group (EADAG), of which I am a member, is one mechanism through which EA seeks feedback from the open data community.
But that approach is inadequate to meet the needs of the many individuals, businesses and community groups who just want permission to re-use specific data or information now.
One of the persistent problems with UK open data is the extent to which release is driven (and constrained) by the institutions that hold the data. Government has declined to legislate for any legal right to secure release of open data. For most users the usual first step to unlocking specific unreleased data is through access to information laws.
EA’s information request process
Environment Agency reportedly receives more than 45,000 information requests a year; that’s more than any other public authority and possibly more than all other UK public authorities combined. Most information requests are handled under the Freedom of Information Act or the Environmental Information Regulations, which provide access to information but only limited rights to re-use that information.
Requesters who want to re-use unreleased data can submit a request under the Re-use of Public Sector Information Regulations, in addition to a FOI/EIR access request. In most cases the public authority is obliged to consider the RPSIR request and respond with a re-use licence. This does not have to be an open licence but the public authority must justify any charge it makes for re-use.
EA’s approach to handling of information requests is efficient but unusual. In order to manage the high volume of requests EA operates a standard process designed to cover the requirements of FOI, EIR and RPSIR at once.
As part of this process most EA releases of data under FOI or EIR are accompanied by a copy of Environment Agency’s “Standard Notice”, a non-open licence for re-use.
EA applies this notice even in the absence of a request for re-use. See for example these recent requests for a list of ESOS-compliant companies and river level peak statistics.
In practice this means EA will evade any request to re-use unreleased data under an open licence (such as the Open Government Licence). EA normally offers only the terms of its Standard Notice.
The EA Standard Notice
The latest version of the Standard Notice was introduced in 2012 when EA made the decision to stop charging for non-commercial use of its information.
The Standard Notice is formatted to resemble the Open Government Licence, but the terms are more restrictive.
Commercial use of information released with the Standard Notice is subject to payment of a £50 licence fee (+VAT) for each person seeking the benefit of the licence.
It’s entirely unclear how that would operate for re-use of data in a product or service delivered via the web or to a consumer market. The Standard Notice is too scant to be taken seriously as a commercial data licence.
The main intention of the Standard Notice seems to be to discourage businesses from mis-using the FOI/EIR request process for profit, and perhaps also to preserve re-use of that information as a revenue stream for EA. But now EA has committed to winding down its commercial data licensing activities, and embraced an open data approach, the Standard Notice is just an unnecessary barrier to re-use of environmental information.
Currently any business that wants to make commercial use of unreleased EA data for a complex purpose is in a difficult position. They can lobby EA for an official open data release, or they can challenge EA’s application of the Standard Notice (£50 per user?) through the RPSIR complaint process. Either approach will take considerable time.
Open by default and the Open Government Licence
I’m always wary of proposing simple solutions. However in this instance I think EA has a clear path towards an open data programme that meets user needs: replace the Standard Notice with the Open Government Licence.
The OGL should not add any additional burdens to EA’s information request process. All the necessary safeguards are already built in to the process of approving access under FOI or EIR. Some data and information will not be suitable for licensed re-use, but if EA can apply the Standard Notice by default it should be able to apply the Open Government Licence by default for requests that meet the same minimum criteria.
EA usually responds to FOI and EIR requests within 20 working days. Information released that quickly will sometimes not meet the technical standards of EA’s productised open data releases. But EA can continue to apply the information warning contained in the Standard Notice. Information warnings are not licensing restrictions and are entirely compatible with the OGL.
The OGL is the UK Government’s preferred licence for re-use of public sector information. Many public authorities now apply the OGL by default to the content of FOI and EIR responses published on GOV.UK. That includes Defra, the government department responsible for Environment Agency.
EA has so far chosen not to disclose its FOI and EIR responses on GOV.UK. In fairness GOV.UK is not an ideal platform for large volumes of material, but if EA does decide to adopt the OGL for user-driven releases of information I hope it will review that position. An EA disclosure log of selected ad hoc open data releases would be an invaluable new resource. (See VOA’s disclosure log and ONS’s ad hoc data for the sort of thing I mean.)
Environment Agency has gone through a considerable amount of organisational and cultural change to develop a creditable open data programme, and I have no doubt it will sustain that effort. Adopting the Open Government Licence by default, rather than by exception as now, is crucial to making the full transition from a commercial to an open model of data re-use.
Image: 43097 nameplate by train_photos (CC BY-SA 2.0)