Post: 28 May 2014 (updated to 3 June 2014)
I wrote in February about the arguments for release of the Environment Agency’s flood-related datasets as open data. This post is an update on developments since then.
Permanent open release of Flood Hack data
The EA released a collection of real-time and near real-time data for three months from mid-February, in connection with the government-backed Flood Hack event. That data release included feeds of data on river levels, groundwater levels and flood forecasts, along with spatial data for flood alert and warning areas.
At the end of April the EA’s data team published a “special notice”, mainly intended for existing commercial licensees, with a list of datasets it intended to make open permanently “over the next few weeks”:
Five of the top six items on that list are those released as open data via the facilities set up during Flood Hack. The air temperature data seems to be an addition; the EA apparently intends to add this data, which is available only for rain gauge sites in the English Midlands, to http://flooddata.alphagov.co.uk/. Groundwater levels data is not mentioned in the EA notice, but that may be an oversight. *
Two of the three WFD datasets are currently available for download from the EA’s DataShare page, but only under a non-commercial use licence. The other WFD dataset, River Waterbodies, may or may not be the same as the River Waterbody Catchments Cycle 2 (Draft) dataset that is already available as open data from the same source. *
Both the GOV.UK page and http://flooddata.alphagov.co.uk/ still carry a notice stating the data there will be available “until May 2014”. The special notice has not been circulated widely, so the EA may intend to make a more prominent announcement about this permanent open data release.
* Updates:
I understand the live feeds currently include data on river levels, flow, air temperature and wind, but that rainfall data was removed on May 15th. So some of the data released during Flood Hack has not been maintained as open data permanently. Thanks to Luke Smith for this information.
Metadata records recently added to the Data.gov.uk catalogue (e.g. here and here) indicate data originating from stations in Wales, included in the live feeds, is not reusable under the Open Government Licence. At the time of writing this is not flagged on the GOV.UK page or on http://flooddata.alphagov.co.uk/.
A new open dataset, WFD - River Waterbodies Cycle 2 (Draft), is now available for download.
Campaign for open flood data - in the news
In addition to coverage of Flood Hack there has been a couple of articles on the Guardian website about the case for further open release of Environment Agency flood data:
Environment Agency poised to open flood data to public (by Charles Arthur)
How open data is being used to hack climate change (by Jack Millner)
NaFRA and other core flood datasets
River levels and other live feeds are invaluable for use in applications that focus on response to emerging flood events. However flood response is only one set of activities. We need to help communities understand the management of flood risk as an ongoing process, and unlocking the EA’s core flood risk datasets can contribute to that.
There has been no official word on plans to release any of the EA’s core flood risk datasets. However I understand negotiations with the Cabinet Office have progressed, and am hopeful there will be an announcement sooner rather than later.
The Environment Agency holds a wide range of flood-related datasets, and my preference would be to see a well-thought-out package of data released under the Open Government Licence.
However the single dataset with the most potential for reuse is likely to be the spatial version of NaFRA. This dataset, an output from the EA’s National Flood Risk Assessment model, classifies the likelihood of flooding from rivers or the sea based on 50m x 50m grid cells:
I have also been arguing in particular for release of centrelines for the river network. Such a dataset would complement both NaFRA and the existing “live” flood open data. (It has always seemed odd to me that we have open data maps of our road and rail networks, but not for the nation’s rivers.)
According to the minutes of a Transparency Board meeting in March, the EA puts its annual revenue generated from licensing of key flood data at around £1.5m. (The figure reported by Charles Arthur in February was “about £1m”, but perhaps that was on a different basis.) This potential loss of revenue is likely to be the main sticking point. However some of that revenue stream may not be sustainable in any case, given expected changes to the existing insurance-sector market for EA flood data (see below).
Flood Re and the Water Act - a new flood risk register
For the past several years the government has been negotiating with the insurance industry to secure the future availability of household insurance in areas at high risk of flooding. At the heart of those negotiations is a reinsurance scheme called Flood Re.
The Water Act 2014, which received Royal Assent two weeks ago, outlines requirements for an important new flood-related dataset: a “register of premises subject to greater flood risk”.
So far we do not know what the government’s intentions are with regard to public availability and reuse of this dataset. However the Flood Re arrangements do seem to mean that government, and the Environment Agency in particular, will become more closely involved in the assessment of flood risk at the individual property level for purposes of insurance cover.
In my view there is a strong argument that the complete register should be released as open data, so that consumers are operating on a level playing field in both the insurance and conveyancing markets. However this will depend very much on the detail of the Flood Re plan. Property data is an area in which, in the UK at least, the dividing line between personal and public information is rather blurred.
The Water Act also includes updated requirements on maintenance of main river maps for England and Wales. The Act enables the EA (and the NRBW) to charge a “reasonable usage fee” for commercial use of the map data. In my view this was a missed opportunity to incorporate open data release of main river maps into legislation.
Flood statistics
In February an analysis produced by the House of Commons Library contradicted government claims about the scale of its spending on flood risk management. This analysis was supported by the UK Statistics Authority, which provides oversight and monitors misuse of official statistics.
In correspondence UKSA chair Sir Andrew Dilnot has said he believes figures on flood defence spending should, in future, be published as official statistics.
I very much hope Defra will take this advice on board. Flood risk, and investment in the alleviation of flood risk, is a subject of considerable public interest. Official statistics about flood risk are notable in their absence.
Ideally I would like to see Defra produce statistics not only on flood spending but also on the number of properties in the undefended floodplain, subdivided using standard administrative geography such as constituencies and wards.
Open data release of NaFRA will enable analysts outside government to produce more outputs of this type, of course. However for purposes of transparency and accountability it would be good to have authoritative statistics from Defra as well.
Photo credit: Morpeth Flood by johndal (CC BY 2.0)
NaFRA screenshot © Environment Agency copyright and database rights 2014. © Ordnance Survey Crown copyright. All rights reserved. Environment Agency, 100026380. Contains Royal Mail data © Royal Mail copyright and database right 2014.