OpenStreetMap

A few weeks ago, I wrote a plugin for PMTiles for JOSM. PMTiles is a web-first tile storage format which allows the provider to have a single file with all the tiles they want to share. Users can then make direct requests using HTTP Range Requests to get just the data they want. With this, other OSM users have discovered that they can run a tile server for <2 USD/month.

As of Aug 28, 2023, the JOSM MapWithAI plugin now supports PMTiles and Mapbox Vector Tiles as data sources. Please note that no tag transforms are occurring, so if you do use the data, please use the replacement tags feature of the plugin. With that said, it is theoretically possible for a pmtiles file to give us the mappings; the specification has a JSON metadata field which can have arbitrary key value mappings.

Anyway, here are some known datasets to test out this functionality (I only tested the overture-pois.pmtiles file):

As a reminder, there is a reason why the overture data has not been imported into OSM, mainly quality concerns. So if you do use it, ensure that whatever it is you are adding actually exists (for POIs) or is geometrically correct (for buildings/roads).

I’m currently looking at what it would take to add them to the default sources.json for the MapWithAI plugin along with known tag mappings, so that users don’t have to follow these steps to add them:

  1. Install/update the JOSM MapWithAI plugin. You may also need to install the PMTiles plugin if it was not automatically installed when you updated the MapWithAI plugin.
  2. Open JOSM Preferences
  3. Go to the MapWithAI sub preference panel
  4. In the Servers tab, there is a Selected entries: section. To the right of that section, there is a + button for adding new entries.
  5. Enter the appropriate url in the Enter Service URL box and change the source type (What is the type of this source?) to PMTILES. You can call the source whatever you like, but I would recommend using something descriptive (such as “Overture Land”, or “Overture POIs”).
  6. Click OK
  7. Download an area from OSM
  8. Data -> MapWithAI -> the name you entered in step 5
  9. Look at the newly added data. You can add it as normal with the MapWithAI plugin. I do not recommend doing this at this time, since the tags have not been mapped to OSM.

If, instead, you just want to browse the overture data, you can install the PMTiles plugin by itself. Once you have done that, you can do the following:

  1. File -> Open Location
  2. Enter the PMTiles URL that you want to browse
  3. Hit enter
  4. (Optional) If you find an area where you want to view the properties of an arbitrary object, there is an advanced option to convert the PMTiles layer to a standard OSM data layer. Please don’t upload the generated layer.

If you find a bug or something is not working as expected, you can file tickets on https://josm.openstreetmap.de/ . Please use the Report bug action when doing so, and change the component to Plugin pmtiles or Plugin mapwithai, depending upon which plugin is exhibiting the issue.

Location: Grand Junction, Mesa County, Colorado, United States

Discussion

Comment from AkuAnakTimur on 15 September 2023 at 04:21

Thanks for your effort to write the JOSM plugin! I’ve tested with Google’s Open Building dataset too, in JOSM, with your plugin as well (someone else is kind to convert and shared a pmtile version of it).

Comment from DarkDays on 15 September 2023 at 08:36

All the Places pmtiles also works, https://data.alltheplaces.xyz/runs/2023-09-09-13-32-00/output.pmtiles

The same reminder as for Overture. Probably don’t use it to add to OSM just yet, unless you’ve validated the data with street level imagery.

Comment from SimonPoole on 15 September 2023 at 10:32

Just for the record: https://github.com/simonpoole/pmtiles-reader and https://github.com/MarcusWolschon/osmeditor4android/pull/2391

PS: IMHO both google and MS building outlines should be considered off limits.

Comment from AkuAnakTimur on 15 September 2023 at 11:43

@SimonPoole

From my observation, Google’s dataset is more rubbish than okay-ish, near where I live.

Comment from SimonPoole on 15 September 2023 at 12:08

@AkuAnakTimur quality is one aspect, but there are other reasons too.

Comment from vorpalblade on 15 September 2023 at 12:19

Just for the record: https://github.com/simonpoole/pmtiles-reader

I probably should have pinged you when I finished the (initial) bits of the plugin back in August (the PMTiles parsing code was done on 2023-08-10); I specifically wrote it so that other users could just import classes from the org.openstreetmap.josm.plugins.pmtiles.lib package without also needing JOSM in the classpath.

As it is, we now have two different implementations for PMTiles reading in Java. I don’t know which is better, especially since I used Java 17 features in the plugin extensively. Which might have been a deal breaker for you anyway.

All the Places pmtiles also works, https://data.alltheplaces.xyz/runs/2023-09-09-13-32-00/output.pmtiles

Good to know. I do accept PRs for the sources.json file I linked to in my post; just make certain you’ve gone through the following import steps: 1. Tag mapping (so no HIGHWAY=ROAD_CLASS_4) 2. License approval. I’ll take links to an import mailing list discussion and/or a license mailing list discussion. 3. Quality review: I don’t want to have garbage data easily ingested. >50% minimum, but ideally 90% or more. a. On the lower end, I’d strongly prefer to filter based off of street-level imagery availability. I don’t think there is currently a good way to do this globally. I wrote a script awhile back that downloaded Mapillary image tracks on a per-county basis that might help with this for smaller datasets.

Comment from SimonPoole on 15 September 2023 at 13:07

@vorpalblade not only Java 17. There are at least 3 different http clients in wide use and I needed to abstract (admittedly in a bit of a hackish fashion) around that too. The thing my library has going for it that it has literally no third-party runtime dependencies so it is very lightweight.

Log in to leave a comment