The redaction bot is a piece of software designed to remove legally problematic data from OpenStreetMap, generally data that violates copyright.

It’s not often needed, as OpenStreetMap users are careful about copyright, but it is available if needed. Requiring special knowledge and API permissions, a grand total of one person runs it regularly – me. This post might not be the most interesting for others…

The code is stored on OpenStreetMap Github and can be cloned with git clone https://github.com/openstreetmap/openstreetmap-license-change.git

Like the API, the redaction bot is written in Ruby. I’m not a Ruby developer, so what I did probably isn’t best practices.

On my machine, I needed to install bundler, and install it locally rather than for the entire system.

1

gem install --user-install bundler

As gem says, I need to modify my path.

1

export PATH=/home/pnorman/.gem/ruby/2.1/bin:$PATH

FreeBSD isn’t the conventional choice for OSM stuff, so my setup varies significantly from others.

I installed devel/rubygem-algorithms archivers/rubygem-bzip2-ruby textproc/rubygem-unf_ext textproc/rubygem-unf www/rubygem-typhoeus textproc/rubygem-libxml-ruby textproc/rubygem-text www/rubygem-mechanize net/rubygem-oauth databases/rubygem-dbd-pg This is probably not the optimal list, but worked.

bundle install --path .bundle might have been of some use too, but I didn’t go that way.

Testing

I started by testing the install with bundle exec ruby test.rb to make sure it ran correctly.

To do the next testing, I needed an account on an API server that had moderator privileges. I have these on http://master.apis.dev.openstreetmap.org/ for test purposes, but I could have also run my own rails port instance.

I started by pre-populating auth.yaml with

1
2
3

---
oauth:
  site: http://master.apis.dev.openstreetmap.org/

Then I ran the script to generate auth.yaml with bundle exec ruby get_auth.rb

I then went to http://master.apis.dev.openstreetmap.org/user/pnorman/oauth_clients/new to register a new application, with map modification privileges and pasted the values into the ruby script, and continued, following instructions. When done, it had populated auth.yaml with consumer_key, consumer_secret, token, token_secret.

Finding some data

Because it’s a test server, I just found a random pair of changesets to redact, 64326 and 64327. Normally this is the hard part of a redaction.

Running the bot

The bot is in the file redact_changeset.rb and can be run with bundle exec ruby redact_changeset.rb, which prints the options.

1
2
3
4
5
6
7

Usage: redact_changeset [options]
    -c, --config CONFIG              YAML config file to use.
    -r, --redaction ID               Redaction ID to use.
    -m, --message MESSAGE            Commit message to use.
    -t, --threads N                  Number of threads to use getting data from the API.
    -n, --dry_run                    Don't perform actions, instead log them to disk.
    -e, --edits_blacklist FILE       edits blacklist file to redact

A suitable call here would be

1

bundle exec ruby redact_changeset.rb -r 1 -m 'test redaction for blog post' -t 4 64327 64326