In August the Cabinet Office's Geospatial Commission unit launched a call for evidence on the National Geospatial Strategy it proposes to develop next year. The consultation closed for submissions last week.
This post contains my responses to the questionnaire, annotated with links and footnotes.
I submitted these responses in an individual capacity based on my perspective as a geospatial data user and campaigner for open public data. All the views are personal. (I also work on open data and information management at JNCC.)
I had a number of "meta" problems with the questionnaire that made it difficult to engage with enthusiasm. To my mind the call for evidence's focus on "innovation, enhancing geospatial assets and driving investment" missed many of the broader societal implications of location intelligence and geospatial data.
The endorsement of Ordnance Survey's "Open OS MasterMap" plan, in advance of any meaningful public consultation and while recruitment of the Geospatial Commission's board was still underway, raises doubts about how much appetite the Commission will have to challenge the status quo on geospatial data infrastructure. As I said in one of my responses below, many of the questions in the call for evidence read like market research for OS's future commercial strategy.
The consultation document provides necessary context for some of the questions.
Q1. Is our view of the geospatial data types accurate? If not, what should be included or excluded from this?
I think the definition for "geospatial identifiers" is too narrow. Many geospatial identifiers are also useful for linking geospatial data to datasets that have no location component themselves, rather than merely linking to "positional data".
The other definitions are fine but this typology is likely to be of limited use for thinking about geospatial data in depth; it may help to develop a more detailed glossary.
Q2. In addition to current government policy, what are the areas of geospatial skills where the commission could best focus, to help ensure the necessary capability within the UK for the future?
Development of geospatial skills may not be an area that requires any significant intervention from the Commission – that should be validated. There's always room for improvement in building data literacy and IT literacy in schools and among the general public, and geospatial skills should remain part of that effort. However I haven't perceived any real lack of skills capacity in the geospatial sector that isn't addressed by existing availability of training.
Q3. What are the geospatial skills needs and gaps in your organisations, how can these be most effectively addressed, and how can careers in the sector be best promoted?
As I'm submitting this response in a personal capacity, I have no answer on organisational needs and gaps. (My current employer may make their own submission.)
I don't know how much strategic benefit there is in promoting careers in the geospatial "sector" per se. Geospatial is a collection of technologies, which have suppliers, and there are careers there. However the real focus should be on the sectors where geospatial data and technology have useful applications. These will have highly varied interests, so the best approach is to demonstrate the utility of geospatial skills in a variety of job roles and contexts.
Q4. Are there any publicly or privately-held geospatial datasets that are currently challenging to access or use or of insufficient quality, but which you or your organisation would find valuable if these issues could be resolved? Please explain why this would be of value, and how access/quality could be improved?
As an open data activist I have a long list of geospatial datasets that I don't think are sufficiently open for access or use. These are a few specific examples:
Flood data. EA made progress a few years ago by releasing its Risk of Flooding from Rivers and Sea (RoFRS) and Flood Map for Planning datasets as open data. However surface water and combined flood risk datasets remain available only on more restricted terms. 1 This creates information asymmetries between consumers and business in understanding flood risk, particularly in conveyancing and in household insurance (where the proprietary data model used by Flood Re is also a barrier).
Land ownership and use. Very little of Land Registry's spatial data is open 2 and the picture in Scotland is even worse. 3 The Rural Land Register is also largely inaccessible. The Commission should revisit the 2017 housing white paper and make transparency of land ownership and interests a strong focus of the strategy.
VOA data remains difficult to access in bulk, especially that related to business rates and floor space. VOA recently and unexpectedly decided not to continue publishing annual small-area statistics on council tax bands and build periods; this reduction in transparency makes little sense given the level of public interest in the state of the housing market. 4
1 Unfinished business from EADAG days.
2 Price Paid Data not open, National Polygon dataset not open, INSPIRE Index Polygons not open, CCOD and OCOD datasets not open.
3 Registers of Scotland (RoS).
4 Actually, the day after I submitted this VOA wrote to say they would publish the council tax data at the end of November.
Q5: Do you anticipate that any changes will be needed to the both address data and the wider address ecosystem, to support emerging technologies? Please provide evidence of value to support any proposed changes.
The current address data "ecosystem" is poorly adapted to support established technologies, let alone emerging technologies.
I don't have any real criticisms of the technical quality of the authoritative bulk address dataset (OS AddressBase), or the GeoPlace system in place for managing the collection and collation process.
However AddressBase is drastically underused in consumer-facing applications (i.e. on the web) due to licensing costs and the investment in security required to comply with restrictions on use.
Beyond the many uses for address data that can only be supported by open licensing, the main benefit from unlocking national address data is likely to be improvements in the quality of the thousands of smaller address datasets collected and maintained by individual organisations.
Currently, cleansing an organisation's own address data (such as a customer list) against AddressBase or PAF 5 represents an input cost as well as a "derived data" risk for the organisation. Organisations tend to only use these datasets to improve the quality of their address data when they have a business case for doing so.
This problem is evident in infrastructure datasets such as the Companies House Register and Land Registry's Price Paid Data, but also affects the quality of thousands of mailing lists, lookup tables, and other address datasets maintained by businesses and charities. Only the availability of an authoritative open address dataset will make data cleansing a standard practice.
Any loss of revenue from licensing of AddressBase and PAF arising from open release of those datasets should be more than offset by economic benefits to organisational users, including efficiencies and new uses that will generate an increase in taxable revenue. In particular we may expect to see a growth in demand for software solutions for managing address data.
There is no shortage of arguments and evidence for the value of unlocking address data. Government has received many representations on this subject; please see the bibliography in this post https://www.owenboswarva.com/blog/post-addr1.htm.
The state of UK address data is a mature debate and at this point the onus must be on Government (and Ordnance Survey / Royal Mail) to demonstrate evidence of value from maintaining the current artificial scarcity in address data.
5 The Postcode Address File, privatised with Royal Mail in 2013; leading cause of radicalisation in the UK open data movement.
Q6: How should the commission be looking to develop the UK's capability in Earth observation data, both technologically and to support an effective market?
The most effective initiative the Commission could undertake in this area is to support the funding of cloud infrastructure for the public dissemination of analysis-ready earth observation data products.
The Copernicus programme is an unparalleled open source of raw data. Many organisations in the UK public and private sectors have the technical skills to exploit this data and combine it with additional data from local and national sources to produce new outputs with specific applications.
However EO outputs are usually very large datasets and few organisations have the resources to store and distribute this data without charging users for egress costs. This limits the potential for frictionless sharing and public benefits.
I urge the Commission to examine the capacity problems in this areas, including potential for developing infrastructure that will support better availability of geospatial data for UK overseas territories and other countries where the UK provides support for sustainable development.
Q7. Which new technologies should the commission focus on to provide new opportunities to process and exploit geospatial data for economic growth?
Ordnance Survey has made a good start with the Zoomstack trial. However I would like to see a more assertive position from Government on data standards for geospatial, and in particular a push to discourage use of the non-open Esri file database (GDB) format. 6
The Commission may also wish to develop a view on metadata standards for geospatial data. Is GEMINI still the right approach, or are there international standards that would increase interoperability in cataloguing of UK geospatial datasets?
6 Obligatory poke at Esri.
Q8. How can geospatial data and applications be used to support enhanced roll-out of future technologies?
No response. I found this question too general.
Q9: What are the options for how public sector organisations could continue to invest in maintaining and enhancing our geospatial data assets?
The Commission should make a clear distinction here between the public sector in general and the small but powerful number of soi-disant public bodies, including most of the "Geo 6", run under special rules that encourage them to exploit public data for commercial income.
As the default model, public geospatial data assets should be developed and maintained from central government funds (or local authority funds if the need is specific to local areas) where there is a business case based on a public task. That public task may be at the organisational level, at the level of the sector in which the public body operates, or based on an identified national interest.
These data assets should also be made available as open data, in order to maximise re-use and value for money from public assets.
Trading funds and similar business models are unsuitable for maintenance of public data infrastructure as they create irresolvable conflicts between public task and commercial incentives, and lead to information economies built on monopoly interests.
This is particularly the case with Ordnance Survey. There is considerable frustration among many developers and data analysts, particularly in web-based startups and the voluntary sector, that the highest-quality datasets maintained by our national mapping agency are not readily available to the public. As one analyst put it, OS is "the great vampire squid wrapped around the face of UK public-interest technology" 7 8 – a massive and unavoidable keystone species 9 in the information ecosystem.
Much of the early enthusiasm for the Geospatial Commission was based on the expectation that government would finally address the OS licensing barriers and other blockages that undermine reuse of land data for planning and housing applications. Unfortunately many of the questions in this call for evidence read as if they are simply market research for OS's future commercial strategy.
On a separate note, as a prerequisite for the investment model I urge the Commission to develop a clear view on which geospatial data assets (existing or conceptual) have primacy as national spatial data infrastructure. Those assets should be monitored and protected by governance rules, similar to National Statistics status, with an expectation that the user community will be consulted on significant changes.
7 It was that Anna Powell-Smith.
8 Merryn owes me a pint.
9 I originally wrote "predator" here. (Scientifically this metaphor doesn't work of course. Vampire squids aren't particularly large or fearsome, or even squids.)
Q10: What areas of the underpinning geospatial infrastructure such as positioning technologies, including GPS and indoor positioning systems, and geodetic networks and frameworks to support them, should we be prioritising the development of, in order to support the emerging requirements for geospatial data?
I would like to see the Commission prioritise ground-based LiDAR, urban sensor webs (for air pollution, traffic management, etc.), and the potential of high-altitude pseudo-satellites (HAPs) for remote sensing.
The Commission should also emphasise the need for ubiquitous and affordable high-speed broadband (both fixed and mobile). Broadband may not be geospatial infrastructure as such but connectivity and throughput are essential to the effective sharing of geospatial data.
Q11: What role should the private sector have in both the development and maintenance of the underpinning infrastructure and enhancing the UK's geospatial data assets?
Underpinning infrastructure for the UK's geospatial data assets should be publicly funded and maintained, with a minimum amount of dependency on private sector participation.
However the private sector should continue to have an important role in identifying user needs and implementing use cases that make use of this infrastructure, and in development of some intermediary services for delivering the benefits to the public.
Where public services are delivered by the private sector, as in transport and energy sectors, the Commission should promote stronger expectations that businesses will publish (either directly or through government) more geospatial data about their operations.
Q12. Do you face challenges when working with geospatial data from across the public sector? If so, what are they and how could value be better released? Are there any technical remedies or standards that could be adopted to improve the interoperability of geospatial data? Please provide supporting evidence of what these remedies could help to accomplish.
Yes. Working within the Public Sector Mapping Agreement (PSMA) framework presents numerous challenges to sharing and use of Ordnance Survey's detailed mapping data, and acts as a deterrent to publication of geospatial data (in particular by local government) that might otherwise be freely available as open data.
Open licensing is the most effective technical approach to improving the interoperability of geospatial data. This is axiomatic; open licences place minimum restrictions on the purposes for which individual datasets can be used, which reduces barriers to use of combined datasets.
Q13. How can the Geospatial Commission act as a more effective customer for geospatial data on behalf of the public sector?
I hope there will be no need for the Commission to maintain this role beyond the short term. I would like to see PSMA/OSMA, the PAF PSL and similar arrangements wound down in favour of a funding model that makes geospatial data infrastructure freely available at the point of use in all sectors, not merely the public sector.
Q14. Are there any additional geospatial datasets, from the other partner bodies or other sources, that the public sector would derive significant benefit from having access to, that might have novel and valuable use cases? What would that access look like?
The Environment Agency, BGS and equivalent agencies in other parts of the UK should have access to detailed data on the location and costs of insurance claims arising from flooding, subsidence, vandalism and other geographic and property-related risks, to assist with economic modelling to support public investment in management of those risks. Some of this data may need to be shared on a commercially confidential basis, but aggregated outputs from this modelling should be published as open data.
As a further example I would also like to see DfE publish catchment area data for schools derived from anonymised pupil location data extracted from the National Pupil Database. This would help analysts understand interactions between school catchments and behaviour in the housing market, among other applications.
Q15: How can we best develop a single UK strategy, ensuring alignment between the individual strategies across the UK while still allowing for regional variations?
No response.
Q16: How can we best ensure effective local authority coordination and sharing of best practise, using location data to better deliver public services? 10
Government should make greater use of the Single Data List to collect and compile national data layers from geospatial datasets held at local level. https://www.gov.uk/government/publications/single-data-list
Government should also mandate use of schemas for returns of geospatial and other data from council sources, to ensure consistency and interoperability.
Replacing PSMA with open licensing of MasterMap data would also make it easier for local authorities to publish and share detailed mapping, for purposes such as planning, housing, road maintenance, and flood management.
10 "Best practise". It's almost like a large American management consultancy had a hand in writing this.
Q17: As a result of this analysis, we are prioritising the exploration of possible initiatives in the high-value categories identified:
What are the existing or potential geospatial applications which could be scaled-up or developed in order to capture economic value? (We would particularly welcome responses from industry and other bodies engaged in these sectors.)
No response. 11
11 Bored now.
Q18: Are there any other areas that we should look at as a priority?
The Commission should look at interactions between publication of geospatial data by public authorities and personal data risks. There are a number of unresolved issues that have not been tested in the courts or examined conclusively by the ICO. For example publication of address-level sale price data by Land Registry, 12 and the recent withdrawal of PARSOL guidance on planning appeals data. 13
There are also potential data protection issues arising from the increased granularity of spatial imagery, as well as emerging technologies like drones and ground-based sensors used to capture environmental observations.
12 I have a blog post on this.
13 This too.
Q19: What are the main potential private and public sector innovations that will rely on the use of geospatial data to rollout, and are there corresponding regulatory challenges?
Connected cars and use of telematics in motor insurance, including as a basis for tailored pricing, will require infrastructure and a regulatory framework that protects privacy and reduces the extent to which individual behaviours will be shaped by insurers.
Rollout of autonomous vehicles, for the consumer market in particular, will rely on the availability of more detailed mapping of the built environment at ground level as well as infrastructure for managing data collected on-the-fly by vehicles.
There is also likely to be a need for more detailed mapping above ground level in urban areas to support the use of small drones including connected drones operating in swarms or flocks. These applications will require better building modelling but also data on weather and wind effects in built-up areas. 14
14 You can tell I read a lot of Wired and Popular Science.
Q20: How best can we make the UK's presence in the international geospatial world more visible?
The UK should maintain involvement in regional and international initiatives related to geospatial data. Where participation has been compromised by EU exit there will inevitably be a loss of influence, but the UK should attempt to introduce domestic initiatives that dovetail with international standards.
For example the UK should continue to support the INSPIRE programme for exchange of compatible public data on environmental themes.
The UK has one of the most complex and sophisticated transport systems in the world. More open data and mapping in this sector could enable the UK to act as a showcase for technology and consultancy services that could be exported to other countries.
Q21: Where should the UK be looking for points of comparison overseas? Who are the other international exemplars? What best practice is being modelled overseas that we can learn from?
No response. 15
15 Apparently the correct answer is Singapore.