Updating bus stops in the West Midlands

posted in: Map Improvements | 0

It has been 8 years since the original NaPTAN (National Public Transport Access Nodes) data was imported into the UK space of OpenStreetMap in 2009. It was done on a piecemeal basis and only in areas where it was requested, which was generally where there were active mappers. Since that time I am not aware of any organised attempts to bring that data up-to-date. There will certainly have been localised efforts based purely on surveys.


So when the local OSM community, mappa-mercia,  was approached by TfWM (Transport for West Midlands) with the offer of resources to bring the NaPTAN data up to date as TfWM was reviewing all their public map production and could consider OSM as a candidate for displaying this data, we leapt at the chance.
We had started out locally some 8 years ago with a lot of enthusiasm and surveyed bus stops crazily but given the scale of the task (some 12,000 bus stops) even a dedicated bunch of mappers is going to get bored – and so we did. Updates and confirmation of existing data languished outside of a couple of public transport enthusiasts and sporadic surveying by others when out surveying something else.
So two internal developers from TfWM and me set to work meeting every Tuesday afternoon in TfWM’s Birmingham head office to put matters right. Our aim was to get an accurate up-to-date map of Public Transport  data in the West Midlands using mostly NaPTAN data. We planned and documented and kept relevant OSM talk lists up to date with what we were doing.
Peripheral tasks were:
1. entering the location of all the new Swift Collector points – these are where passengers can upload new ride allowances into their Swift Smart Cards
2. entering West Midands Fare Zone data to all railway stations in the region
3. entering the proposed routes of all planned Metro tram lines
The first task was to clean up the existing OSM data. This involved moving bus stop nodes that were nodes on highways rather than to one side, identifying and tagging non-naptan bus stops  ( for example car park shuttle buses at the Airport, large campuses such as the National Exhibition Centre and personnel shuttle buses for multisite campuses, usually hospitals) and finally identifying, and resolving where possible,  what we dubbed “orphan” bus stops which were those surveyed by OSMers but with no NapTAN data attached.
The next task was to check the positional accuracy of the current 2017 NapTAN data against both aerial imagery and known OSM well-surveyed areas, as that would determine our approach. Sadly we found that the positional accuracy had not improved systematically over the 8 years. So any refresh of NapTAN data would have to rely on OSM position.
Our next step was to remove all those bus stops marked DEL, for deleted, in the NaPTAN data by removing the highway=bus_stop tag and adding a note to that effect and leaving the remainder of the tags. This was very much a belt-and-braces approach.
Next we updated all the bus stops that had a NapTAN identifier with a standardised naming convention preferred by TfWM and agreed by the local OSM community,  and with current route_ref information.
That left all the new bus stops that had been added in the intervening 8 years. This had to be done semi-manually in order not to remove any bus stops surveyed by OSMers and also to check the positional accuracy (OSM surveys always won!)
As CUS (Customary) stops do not render as there is nothing verifiable on the ground such as a pole we imported those first, as is.
Then any remaining “orphan” bus stops were checked against the new bus stops, relevant data transferred and positions moved before deleting them. We managed to remove nearly 300 of these but were unable to resolve about 50. Then the new bus stops with improved positions were imported. We could have automated this more, but preferred to adopt a more cautious , if laborious, approach of human review.
Finally, having achieved an accurate up-to-date bus stop estate we added shelter information: shelter=yes and shelter_ref=xxxxxxxx.
Loose ends are largely those bus stops still in existence on the ground with NOT IN USE displayed prominently on the pole flag, which NaPTAN treat as non-existent but OSMers can see and will map; and Ring and Ride bus stops which also were  not in the NapTAN data. There are also numbers of bus stops with poles that are still labelled as CUS in NapTAN.So we never achieved 100% accuracy but the percentage of errors/queries is of the order of <1%.
Next we have to agree with TfWM not to let our hard work atrophy by entropy  but keep the data up-to-date by agreeing a frequency of when we do it and a process of how we do it.
What we didn’t do: upgrade existing  and enter  missing bus routes as relations. Neither party was willing to devote the resources to entering these as relations. Which was a pity but maybe we have to realise that the easiest way of representing these given their quantity and volatility ( and the upcoming changes in the Bus Bill which will demand electronic submission of bus routes) is layering on the base map in a separate application.
The process in numbers

Item Approx number
Elapsed time 3 months
TfWM resource hours 30 hours
OSM resource hours 30 hours
Total  bus stops 12,500
Bus stops deleted 400
New bus stops imported 800
Orphans removed 300
Shelters added 5,000

 

Conclusion
TfWM have some great developers and we quickly developed a rapport together. We have built good relations not only at a personal level, but also as organisations.
If the state of our bus stop data in the West Midlands is typical for the UK then we have a lot of work to do to obtain an accurate UK-wide map of bus stops. If we were to  contemplate upgrading throughout the UK I would say it couldn’t be done without a dedicated full-time team and would consequently need some national sponsorship with a business case for devoting the resources necessary. A region-by-region or city-by-city approach would be more doable. So how about it OSM UK and DfT (Department for Transport)?

Massive Release of Highways Asset Data in Birmingham

posted in: Uncategorized | 1

This month Amey released a number of  open datasets to Birmingham City Council, which has published them on its Data Factory site under an OGL licence

Amey maintains Birmingham’s highways under a 25 year Highways Maintenance and Management Service, which is Europe’s largest local government highways partnership.

It covers 2,500km of roads, 4,200km of footways, 95,000 street lights, 76,000 street trees, around 1,100 traffic light signals and over 1,000 bridges, tunnels and highways structures. Winter services involve gritting 1,200km of road every night during freezing weather, maintaining 1,265 grit bins and treating priority pavements in icy conditions.

The datasets cover: trees, traffic signals, streetlights and gullies (points where surface water drains off the highways) as at December 2016

The tree dataset seemed the most interesting to us, especially with relation to other datasets for air pollution that are being generated. A review of the data showed its positional accuracy to be pretty good: none of us have specialist knowledge about tree species so we are accepting the accuracy of Amey’s data.  Amey’s data contains details of the species, estimated age and shape of the trees as well as other identifiers.

Currently we’re engaged in importing the tree dataset which we’re doing on an area basis so that we can do a human review and delete existing trees which have been added from various aerial imagery sets. One of the benefits of  this method is to eliminate trees in the Amey dataset that are identified as “assets to be de-accrued” – these refer to trees that have been removed either because of highway improvements, storm damage, disease or safety.

It’s a great shame to pass a highway junction where you’ve just imported the trees to see the tree surgeons at work felling them all in preparation for a junction improvement!

We have of course raised with the Amey the question of how the dataset is to be maintained as open data. It would be such a shame for our hard work and their welcome initiative to degrade over time because there is no mechanism for updates of additions and deletions.

We’re not sure yet what to do with the other data. The traffic signal data is probably the next most interesting. We’ve taken a brief look at this and our current thinking is to import the data as untagged nodes and gradually manually transfer the UIDs to adjacent traffic signals, deleting the imported node. We’re pretty confident that with previous work we’ve done in collaboration with Birmingham City Council we’ve captured most of the traffic signals; and any way trees are going to keep us occupied for a few weeks.

We’ll keep you posted.

Mysterious Objects: no 7 in an occasional series

posted in: Observations | 2

It’s been a long time since I’ve found an object on my surveys that has left me perplexed. Neary two years actually. The cynical side of me thinks its probably I haven’t been curious enough or motivated enough to document it. And to my embarrassment, it’s on my daily walk to the shops.

Anyway, can anyone help with this one? It’s about 3 inches square and embedded in a pavement and doesn’t look like it’s  a lid that can be opened to something like a valve or switch. Also it looks like it’s brass which is unusual for street furniture of this kind.

There’s no map to look at in this blog- it’s secret!

posted in: Observations | 4

The attempts at secrecy surrounding the UK’s water supply became even more bizarre following my previous blog concerning the location of fire hydrants in the West Midlands, where making a map of fire hydrants is considered a threat to national security.

Now we have the local water utility company, Severn Trent Water, joining the West Midlands Fire Service in attempting to prevent mapping of water resources, using similar “national security ” concerns, although the phrase used by STW is “national critical infrastructure”.

As part of the current OSM UK quarterly project which is to improve mapping of any water features I thought it would be great to map the installation of the new pipeline paralleling the existing Elan Valley Aqueduct which is imaginatively named the “Birmingham Resilience Project”.

BRP is a massive  £242m  two-year civil engineering project and is designed to  provide resilience to the Edwardian aqueduct opened in 1904 which supplies Birmingham all the way from a reservoir in the Elan Valley in Wales.  This pinnacle of Edwardian engineering is  a 73 mile (117 Km) supply which requires no pumps, relying on gravity and siphons.

The Birmingham Resilience Project will provide a parallel supply for the last section of the aqueduct. It will run 16 miles (25 Km) from the River Severn at Stourport to the Frankley Reservoir to the West of Birmingham. It will be  a pumped underground pipeline. The best short online description of the project is here. There’s also an in depth presentation here  by the Head of Pipelines from the main contractor Jacobs

STW’s informative online pages about the project include a FAQ  where you can apply to look at the map of the proposed route(!)

Originally STW had published an online map showing the proposed route overlaid on an OpenData map from the Ordnance Survey (the UK’s national mapping agency). This would have made mapping it for OSM a doddle. However there was a statement on this map:
“This drawing is not to be used in whole or part other than for the intended purpose and project as defined on this drawing.  Refer to the contract for full terms and conditions.”

So I contacted STW to see if we could use the map as I couldn’t refer to the contract which isn’t a public document. (Remember all  UK public utilities, except the National Health Service and the BBC, and large swathes of central and local government are run privately with no accountability or information other than to contracted parties or shareholders.)

What transpired was a phone conversation which was quite stunning in its convoluted reasoning. I didn’t record it, but it was along the lines of:

“Yes we’ve published a map based on opendata but no you can’t use it”

“Why publish it, if we’re not allowed to use it?”

“We need to undertake public consultation, but the data can only be viewed not replicated as it concerns national critical infrastructure”

“So if I walk along public footpaths and roads and take photographs and gps readings wherever I can see this 16 mile civil engineering project and then make a map from my collected data, that woud be OK?

“Yes I guess so”

“So why not save me the effort and give me the data, which has already been made public, as the effect will be the same: publishing a map of a highly visible civil engineering project?”

“We can’t do that – it would compromise national critical infrastructure”

Repeat the last two statements ad infinitum.

Despite its absence from STW’s website and the medieval process of applying to look at it (and no doubt have your name recorded as a potential threat to national critical infrastructure); for those of you who want to see  a map of the proposed route, the online planning sites of each local authority that the new aqueduct traverses (Wyre Forest District Council, Wychavon District Council, Bromsgrove District Council, Birmingham City Council) provide dozens of very detailed maps, all of which are copyrighted however. So it doesn’t get us any further forward other than having to do a ground survey, but it makes a nonsense of STW’s attempts to make online access to maps difficult.

In STW’s parallel corporate universe you can have a two-year civil engineering project digging up 16 miles of the countryside with its attendant huge online archive of public planning documents all of which you can make invisible! Just as long as you don’t have an online opensource map!

Anyone feel like helping with the ground survey?

 

 

A guide to mapping Fire Hydrants in the UK

 

In the West Midlands our fire hydrant signs are generally placed on lamp posts:

The black H on a yellow background I believe to be a UK standard. The upper number is the diameter of the underlying water main in mm. The lower number is the distance in feet to the fire hydrant from the sign with the arrow showing the direction to the actual hydrant. There is also a reference number at the foot of the sign. So, having spotted the sign, the actual fire hydrant has to be found. The signs can get swivelled on the lamposts through maintenance interventions or general neglect and the arrows point in the wrong direction!

To complicate matters further, there are some fire hydrants that do not have a sign, and some older variants of the signs. These older signs tend to be attached to buildings or walls and I have no idea what the numbers refer to.

So you have to on the lookout on the ground as well as spotting yellow H  signs.Fire hydrant covers generally have FH on them to identify them, although some older ones can have just H only. Whether the older style ones are still operational I don’t know, but they get mapped anyway.

 

And of course it’s always good to see public organisations collaborating successfully with each other!

The basic tag is emergency=fire_hydrant with more details on the OSM wiki

Why the sudden interest in fire hydrants? In the UK, they’ve languished as an item that gets mapped.

At our last mappamercia pub meeting Andy Mabbett regaled us with his saga of trying to get West Midlands Fire Service to release the locations of fire hydrants under Freedom of Information legislation.

The full saga can be found here

WMFS refused to release the information on the grounds of national security viz: “publishing information about water  networks and other parts of the critical national infrastructure could  expose vulnerabilities in the network and pose a serious risk to public health either through non availability of water resource or
contamination of supplies. There does not have to be any evidence that this is being planned but it is a possibility given the current threat level in the United Kingdom.”

An appeal by Andy against WMFS to the first-tier information rights tribunal mainly on the grounds that other Fire Services in the UK had already released their information on fire hydrants was rejected. The judges agreed with the Information Commissioner’s Office (ICO) assessment:

“The ICO concluded that the withholding of the information was reasonably necessary for national security and a listing of hydrants and their locations would assist in the planning of an attack by poisoning on water supply infrastructure by identifying access points. Such an attack on the infrastructure would be in the domain of national security. While hydrants are visible a comprehensive list of the precise location of every hydrant would place in the public domain more information than is available through hydrants being visible. WMFS supplied a list of attacks and attempted attacks on water supplies.The ICO considered that such an attack was plausible.”

There is a method for contaminating the water supply known as backflow contamination, which is considered to be reasonably easy according to an excellent paper here which quotes extensively from US military and homeland security sources. Fire hydrants are one possible source to generate backflow.

My limited understanding of water networks is that backflow is a general problem for the water supply industry, which can mitigate the effects with backflow prevention devices. Just how extensive and successful this mitigation is in the UK (and thus how safe we are from backflow attacks) will remain a commercial mystery, as our water supply is in the hands of private companies who are under no obligation to reveal such information. ( I have another story about water networks and open information which will be the subject of a later blog)

Given the cult of secrecy that exists at all levels of official Britain, it also seems unlikely there will be any disclosure of the risk levels of such an attack vector so that we might make up our own minds based on the data. We will have to make do with the assurances of “those that know best”. The public’s role seems to be limited primarily to that of potential or actual victims.

This issue does raise some interesting ethical challenges for OpenStreetMap as it seems to be sending us back several centuries when accurate maps were regarded as military secrets. Or to the days of the Cold War when our national mapping agency the Ordnance Survey would obligingly leave blank spaces on their maps at the sites of military installations. Would a map of fire hydrants in the West Midlands be construed as offence under ss57&58 of the Terrorism Act 2000: collecting, possessing or making a record of information of a kind likely to be useful to a person committing or preparing an act of terrorism? Should we be standing up for opendata, one of the founding principles of OpenStreetMap, or protecting  (supposedly) national security? What is the position in other jurisdictions where there are more draconian restrictions about identifying and mapping military installations and “national critical infrastructure”? Are mappers more circumspect there?  What’s the legal postion- is the OSMF liable for prosecution or just the contributing mapper or both? Should OSMF comply with a demand to remove information on the grounds of national security? Or to refrain from collecting it in the first instance?

According to taginfo there are only 1786 fire hydrants mapped in the UK, of which there are now about 300  in the West Midlands. Prior to my interest being piqued by this sorry tale there were only about 5 fire hydrants mapped in the West Midlands. Judging by the density of fire hydrants I’ve discovered so far, there are probably thousands in the West Midlands so I seriously doubt whether we’ll ever crowdsource the location of all of them,(or even many more than we have already).

Nonetheless, it’s been an education in another aspect of urban infrastructure I wasn’t really aware of, and a confirmation of the patrician “you don’t need to know about that” attitude of much of British officialdom.

(all of the images are my own and are published here as public domain)

.

OpenStreetMap UK Local Chapter officially exists

posted in: Participate | 0
On Saturday morning I received from Companies House the Certicifate of Incorporation for OpenStreetMap United Kingdom Community Interest Company Ltd. It is a private limited company, limited by guarantee.

It’s taken us a year to get this far, through a protracted and  often tedious process of agreeing Memorandum and Articles of Association,Community Interest Statement, form-filling, and signature gathering. Thankyou to everyone who participated, especially the volunteers who agreed to be the first directors necessary to get the thing off the ground.

Now we can start doing the fun stuff of how we make this work and transform it into a living organisation.(Although I’m sure we’ll still have our share of bureaucratic process to navigate).

Prizewinners! Birmingham Highways Data Challenge

posted in: Mapping Party | 1

Mappa Mercia was at the 2 day Birmingham Highways Data Challenge hackathon last weekend where numerous datasets relating to highways and transport were made available. We won a prize for developing  an HS2 HGV Traffic Heat Map. It’s not the finest visualisation, but we were up against a time constraint, poor initial data (essentially text descriptions of routes with no geolocation data) and having to learn new tools from scratch.

hs2-heatmap

Link to Visualisation

Judges remarks

A striking example of an entry that demonstrated shortcomings in the manner in which data is published by the organisation that some members of the team worked for, leading to them finding a fix for this which they will take back to their organisation; in this case HS2, with a recommended, enduring fix. Nothing like a bit of user testing to highlight improvements and a swift response by HS2 data managers to find a solution.

Team

Andrew Gaitskell (HS2) and Brian Prangle (OpenStreetMap / Mappa Mercia)

Description

Traffic heat map showing anticipated HS2 HGV traffic intensity and flows – for the purpose of visualising the data the developers chose forecasted HGV traffic flows around Birmingham International Airport arising from the siting of HS2 construction compounds.

Technology

ArcGIS to extract CSV data of Traffic Link and HS2 Compound Location Data from GeoDatabase file downloaded from eB (HS2’s document management system)

Online XY to Lat,Long coordinate converter. http://gridreferencefinder.com/batchConvert/batchConvert.php

Excel to manipulate data and match XY Conversions to Traffic Flow Data

OpenStreetMap for base map

Used https://carto.com/ to display the location of the compounds and Traffic flow data

Data used

Machine-readable extracts from published documents.

Additional data was required to identify the planned HS2 Compounds

GeoDatabase Files of HS2 Traffic Links and Compound Locations.

Prize

Raspberry Pi 3

 

Transport for West Midlands: Public Art

posted in: Use The Map | 0

Scattered around the West Midlands at bus interchanges are numerous sculptures, commissioned by the transport authority, Centro. They were commisioned and erected in the early optimistic years of the current century. They are quite striking and try to reflect the history and local culture of the community which use the bus interchanges.  Bus interchange sounds a grand title, but on the ground they’re mostly collections of bus stops, where you need to change bus routes.

Not every bus interchange has a sculpture and I don’t know what the criteria were for choosing them. I guess the money ran out before completion.

Centro no longer exists, having changed its name to Transport for West Midlands, upon the creation of the new tier of local government, the West Midlands Combined Authority.

I decided to make a separate map of these artworks, even though they can be found in the West Midlands Heritage map, where they are just another artwork amongst hundreds of others. They are a unique collection of public artwork and are regularly photographed for wikimedia,flikr and the like. So if you see these photos and wonder where the artworks are, there’s no map to find them. So a dedicated map seemed the best solution.

tfwm-artwork

I use umap to create the map and found it incredibly easy, and  I will certainy be creating more maps with umap. It took at least ten times longer to ensure the data was correct.

Congratulations!

posted in: Participate | 1

Congratulations to everyone who has participated in the various UK Quarterly Projects. We were nominated for an award at the recent international State of the Map 2016 conference in Brussels.

Unfortunately we weren’t awsome enough to win, but it’s nice to be recognised, even though that’s not why we do things. It’s just fun!

Thanks to everyone who voted for us.

Anyway here’s  the nice certificate that Rob brought back from Brussels

osm-award

Summary of the Summer UK Quarterly Project

Our Summer project  which ended on 30 September was to map as many farmyards as we could.
1. We added 1499 farmyards, which compares to an increase of 853 schools for the schools project which seems to suggest farmyards was a more active project, but the schools project added 3653 polygons( i.e we converted lots of nodes to polygons, thereby enriching the data). So this time we created more data, rather than improve existing data.
2. We gathered momentum during the project: it took 39 days to edit the first 500 farmyards; 33 for the next 500; and 20 days for the ultimate 500 (well, 499)

3. There are over 19,000 place=farm tags, almost all of them (>18,000) nodes. Mostly they seem to indicate farms but sometimes they get used too enthusiastically for any group of buildings. It is generally agreed that this tag really doesn’t add any useful information and its use should be discouraged.

4. We have 36 generically named Poultry Farm or Poultry Houses copied from OS OpenData, which describes the farm but doesn’t actually name it!

5. A lot of farms in Herefordshire (which is where I mapped mostly for this project) don’t have the word Farm in their name. Is this the case elsewhere?

6. If Herefordshire is anything to go by road alignment in rural areas can be pretty poor.

7. Waterways traced from NPE are severely misaligned.

8. 5 and 6 might suggest themselves as future quarterly projects. Correcting them around farms I added certainly slowed down my output.

9. Don’t ask farmers to help you mapping during their busiest time of year! I got zero response from my approaches.

10. Most unusual name I found was Cold Comfort Farm. There are 4 of them in the UK and had a comic novel named after them

Herefordshire farmyards before and after:

hereford-farms-june hereford-end-sep