My last post was about about how to install Avecado and that it met the needs for some OpenStreetMap Carto benchmarking, I didn’t cover how to set it up with a stylesheet. The basic idea is simple, but some adapations need to be made so the benchmark workload matches a real-world workload.

The avecado_server program has a simple built-in HTTP server for testing purposes that can be used with a HTTP client like curl to produce a load on the database. It lacks sophisticated caching but any caching would need to be disabled for this benchmarking.

Setting up a stylesheet

To start, we need a stylesheet. Since it’s what I’m benchmarking, this is OpenStreetMap Carto.

1

git clone https://github.com/gravitystorm/openstreetmap-carto.git

The SQL queries for each layer are defined in project.yaml, along with the zoom range where they are used.

One difference between a vector tile rendering stack and the typical raster rendering chain is that raster tiles are rendered in blocks of 8x8 tiles called meta-tiles. One meta-tile covers the same area as a tile from three zooms lower, so we can request the right area by lowering the zoom by three, but we have to make sure the right layers are used. This requires changing the zoom ranges in project.yaml.

There are sophisticated ways to do this, but a simple way is to replace the strings for min and max zoom

1
2
3
4

for z in `seq 3 18`; do
sed -i -e "s/^      minzoom: $z/      minzoom: $((z-3))/" project.yaml
sed -i -e "s/^      maxzoom: $z/      maxzoom: $((z-3))/" project.yaml
done

The number of spaces is significant!

Getting data

To be able to render anything, data is needed. Switch2osm has an excellent guide to loading data with osm2pgsql. For the benchmarking I was doing, I loaded the full planet file from 150202. Besides a database, some shapefiles are needed, and can be downloaded with a script included with OpenStreetMap Carto. This script requires shapeindex, a utility from Mapnik, so PATH and LD_LIBRARY_PATH need to be specified to use the Mapnik install from earlier.

1
2
3

sudo apt-get install unzip gdal-bin
cd ~/openstreetmap-carto
PATH=$HOME/mapnik/bin:$PATH LD_LIBRARY_PATH="$HOME/boost/lib:$HOME/mapnik/lib" ./get-shapefiles.sh

Mapnik XML and Kosmtik

Avecado renders vector tiles using stylesheets written in Mapnik XML. This is a very difficult format to work with, so instead most stylesheets are developed in CartoCSS, which adds a pre-processing step. The required pre-processing can be done with Kosmtik.

Kosmtik is a node module, so requires installing NodeJS and NPM to use.

1
2
3
4
5

sudo apt-get install nodejs nodejs-legacy npm
cd ~
git clone https://github.com/kosmtik/kosmtik.git
cd kosmtik
npm install

Turning CartoCSS to Mapnik XML is done with Kosmtik’s export task

1
2

cd ~/openstreetmap-carto/
node ~/kosmtik/index.js export --format xml --output openstreetmap-carto.xml project.yaml

Running avecado_server

For the next steps, it helps to have a terminal multiplexer like tmux or GNU screen, or multiple SSH connections.

In one window, run avecado_server

1
2
3

LD_LIBRARY_PATH="$HOME/boost/lib:$HOME/mapnik/lib" ~/avecado/bin/avecado_server \
  --thread-hint 8 --max-age 0 --buffer-size 16 --port 8080 \
  --map-file ~/openstreetmap-carto/openstreetmap-carto.xml

There’s a few parts to this command

• –thread-hint specifies how many threads to run in parallel. Generally this should be the number of threads the CPU supports, but this varies when benchmarking

• –max-age 0 gets rid of caching

• –buffer-size sets the size of the buffer. The buffer is important for having consistent labels across meta-tiles at boundaries so the labels do not get cut off. Because we are rendering tiles three zoom levels lower, we divide the standard buffer of 128 by 8.

• The other options tell avecado_server where and what to serve.

On startup, avecado_server will complain about missing fonts. If we were intending to render raster tiles in Asia, this might be an issue, but it doesn’t matter for benchmarking.

Testing

In another window, we can download a tile to test that everything is working. We can do this with

1

curl -s --compressed 'http://localhost:8080/15/5195/11223.pbf' | strings -n8 | less

Running this we should see a bunch of layer names, and keys and values for rendering. The tile requested is a zoom 18 meta-tile in uptown New Westminster. The --compressed option makes curl request compressed content and transparently decompress it.

To work around bugs in MapBox Studio, avecado_server defaults to returning compressed content, so without the option the downloaded tile would need to be passed through gunzip.