Have we hit the gem that will improve OpenStreetMap’s completeness

15 07 2008

OpenStreetBugs (http://openstreetbugs.appspot.com) is a wonderful application created by Xavier in Rennes, France which allows us to tag areas in OpenStreetMap which are incorrect, incoherent or incomplete. Its an idea I’ve been talking about since my project began, little did I know that this little gem was around, created about a month ago as can be seen from Xavier’s blog. If we all start to use this app I’m sure it will prove invaluable in our quest to make OSM complete.

Users can annotate OSM in pretty much anyway they want to, others can then come along and discuss the notes. Or even open Potlatch centered on a node or download a GPX dump of the visible nodes to load in JOSM (or in your GPS). With recent updates you can even obtain an RSS geo-feed for an area that you choose and monitor changes.

If OSB does become popular then it can become the one-stop shop for inaccuracies in OSM. People will be able to check how many flags there are in a particular areas and from that get an idea of how complete the map is in that area. It will also act as a guide for avid mappers who don’t know where to start next, just pick the area with lots of incomplete tags. Moreover we can encourage people to participate in the OpenStreetMap project just by visiting OSB and tagging everything they know is incomplete. That way people who feel a little bit frightened by the whole concept of going out mapping can still help improve the map. Perfect!

New project: Lets use LandSat

10 07 2008

Not sure about the licensing requirements for my beautiful Qgis model (which shows roughly which areas of the UK are complete and which incomplete) so I don’t think I can publish the full results from that yet. Essentially though I have found that there is a strong correlation between the length of road in a particular area and the population in that area, its good enough to accurately predict the length of road there should be in an area and compare that to OSM road length. But enough about that project for now, hopefully soon I will publish all my findings.

So instead its onwards and upwards to a new project which was inspired by the view of the Osmarender layer on OpenStreetMap, shown below. It is clear to see that there are vast areas in Asia and South America, where there is no OpenStreetMap data, the question is whether there is actually nothing there or OSM is just missing cities, roads etc. I plan to find out using open source aerial imagery.

The plan is to use LandSat and other forms of freey available imagery to work out where there should be cities and roads and where there shouldn’t be. Then take this information and compare to OSM. Easier said than done, I’m sure but it wouldn’t be a project if it wasn’t challenging. So below is a LandSat image of London and to the North, which I have applied a Yellow Contrast Gradient Map to, using GIMP. As you can see it emphases cities and rural areas quite well and I’m sure this is the starting point to predicting accurately where there should be OSM data.

Qgis Problem Solved

7 07 2008

I’ve moved on to using the models I generated with Local Authority DfT and ONS statistics to predict which areas of the country are complete and incomplete. I used the model on Lower Layer Super Output Areas (LLSOAs) which have an average population of 1500 and variable areas. Using Qgis I wanted to create a ratio of OSM road length in an area divided by the length of road the model predicts. In this way areas that have a value of 1 are complete, values under 1 show that there is not as many roads in OSM as I would predict and are therefore incomplete. Unfortunately there are a few areas with values larger than 1, indicating that the model is under predicting the road length. After extracting OSM road length for every boundary using Qgis I used OpenOffice spreadsheet to apply the model to every boundary, the problem came in re-importing the data to the shapefile for use in displaying heat maps in Qgis. The only way my colleague could find, involved a serious amount of hacking and command line stuff, which I am not very fluent in. Luckily with a bit of a search I found this solution.

All of the attribute data for the shapefile (i.e. all the data apart from the coordinates) is contained in a dBASE (.dbf) database file. Now if you attempt to open this file up as a database file in OpenOffice, ie by right clicking on it and make you open with OpenOffice Base then it will load up in OpenOffice spreadsheet in the correct file format. Select Unicode (UTF – 8 ) as the character set in the pop-up window and you’re good to go. You can manipulate the data in whatever way you like just be sure to save it as the same file name in the same file format. Then when you import the shapefile to Qgis it will contain all of your new attributes and you can make some new fancy heat maps as I intend to do.

Pretty soon I hope to have some nice image outputs from Qgis  which will show which areas of the UK are complete and which incomplete. Keep checking back for that and ask as many questions as you can muster, I’m always interested to hear from readers.