Paul’s Blog

A blog without a good name

OpenStreetMap Standard Layer: Requests

This blog post is a version of my recent SOTM 2021 presentation on the OpenStreetMap Standard Layer and who’s using it.

With the switch to a commercial CDN, we’ve improved our logging significantly and now have the tools to log and analyze logs. We log information on both the incoming request and our response to it.

OpenStreetMap Standard Layer: Introduction

This blog post is a version of my recent SOTM 2021 presentation on the OpenStreetMap Standard Layer and who’s using it.

The OpenStreetMap Standard Layer is the default layer on openstreetmap.org, using most of the front page. It’s run by the OpenStreetMap Foundation, and the Operations Working Group is responsible for the planning, organisation and budgeting of OSMF-run services like this one and servers running it. There are other map layers on the front page like Cycle Map and Transport Map, and I encourage you to try them, but they’re not hosted or planned by us.

OpenStreetMap Survey by Visits

In my last post I looked at survey responses by country and their correlation with mappers eligible for a fee waver as an active contributor.

I wanted to look at the correlation with OSM.org views. I already had a full day’s worth of logs on tile.openstreetmap.org accesses, so I filtered them for requests from www.openstreetmap.org and got a per-country count. This is from December 29th, 2020. Ideally it would be from a complete week, and not a holiday, but this is the data I had downloaded.

Preview image

The big outlier is Italy. It has more visits than I would expect, so I wonder if the holiday had an influence. Like before, the US is overrepresented in the results, Russia and Poland are underrepresented, and Germany is about average.

Like before, I made a graph of the smaller countries.

Preview image

More small countries are above the average line - probably an influence of Italy being so low.

OpenStreetMap Survey

The board has started releasing results from their 2021 survey. I’ve done some analysis on the response rates by country.

There’s lots of data for activity on OSM by country, but for this I took the numbers from joost for how many “active contributors” there are according to the contributor fee waver criteria.

Preview image

For the larger countries, Russia is the most underrepresented country. This is not surprising, as they are underrepresented in other venues like the OSMF membership.

The US and UK are both slightly overrepresented in the survey, but less so than I would have expected based on other surveys and OSMF membership.

The smaller countries are all crowded, so I did a graph of just them.

Preview image

As with other surveys, Japan is underrepresented. Indonesia, although underrepresented is less underrepresented than I would have expected.

OpenStreetMap Cartographic: A Client-side Rendered OpenStreetMap Carto

I’ve been working on a new project, OpenStreetMap Cartographic. This is a client-side rendering based on OpenStreetMap Carto. This is an ambitious project, as OpenStreetMap Carto is an extremely complex style which shows a large number of features. The technical choices I’m making are designed so the style is capable of handling the load of osm.org with minutely updates.

I’ve put up a world-wide demo at https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html, using data from 2020-03-16, and you can view the code at https://github.com/pnorman/openstreetmap-cartographic.

Creating Tarballs

With all the tiles generated and optimized, they just need to be packaged in a tarball. Before creating them, we want to create some files with metadata about what was used to generate the tiles. The commit of the stylesheet and the timestamp of the planet file can be extracted with a couple of commands.

Optimizing PNGs

With the tiles generated normally the next step would be to serve them, but because I’m planning to distribute them to others, I’m going to the unusual step of optimizing the PNGs. Optimizing PNGs can cut the file size in half, helping downstream users of the tiles I’m generating.

Seeding Tiles

With the database loaded, all the software installed, and everything configured, it’s time to render tiles. This is done with the mapproxy-seed program, using the previous config files. The only option needed besides config file locations is -c which sets how many CPU threads to use. For the machine I’m using, 7 works best. Fewer leaves some capacity idle, while running with too many threads starves PostgreSQL and system of any CPU time.

Configuring MapProxy

MapProxy needs a couple of configuration files. One defines the layers, caches, and services that it runs. The other is used for “seeding” the cache, and specifies what to pre-render.

There’s a lot of documentation on MapProxy configuration files, and example ones can be created with mapproxy/bin/mapproxy-util create -t base-config.

More Work on Bolder

After the birds of a feather session Richard Fairhurst lead at State of the Map, I was motivated to continue some work on bolder, a client-side style I’ve been working on.

While I was working at the Wikimedia Foundation, I developed brighmed, a CartoCSS style using vector tiles. Wikimedia decided not to flip the switch to deploy the style, but the style is open source, so I can use it elsewhere. Making this decision, I spent a day implementing most of it in Tangram.