So the SotM EU just ended (guys from UK asked to call it “Europe”, not “EU”). We had a lot of talks, hundred people from the States and Belgium, met many friends. And also learned a bit more about Overture Maps. Nothing new since my Shtosm post, but updates my post about donating money. This is a rough DeepL translation of this and this telegram posts.

Marc’s Address

First, a simple one. Overture is not OpenStreetMap. Marc started by saying that the target audience for Overture Maps is developers. Just as Ballmer once chanted: developers, developers! And he’s right: Overture makes working with data much easier for developers. The data is collected, cleaned, in a convenient format, take it and build it into the product.

So the audience is product developers. Who know little about geo, but a lot about building products. As I wrote in the reddit about VLC, there are developers and there are developers. The audience here are opensource developers: god knows how to organize them. They are not the TA of Overture. They’re doing OSM. So when Mark encourages developers to use (and of course improve) Overture here, he’s kind of leading developers away from OSM. And that’s the first thing that was a bit tone deaf in his presentation.

The second is what I detailed in my question after the talk. That is, I’ve written before that Overture is an awesome wrapper to OSM. It sells data, it sells an idea, it does all the things we don’t want to do. But at the same time, the attitude towards Overture and the community was: you’re doing a great job, all these maps are very good, the tools you’ve written are great too, keep it up.

A star for the effort. OSM is some kind of a magic project that miraculously creates very good data from which to build Overture. I mean, I’m not against it, I like the idea. Overture is easy to sell: Esri, TomTom, Meta, Microsoft and others immediately jumped in for a reason. Overture has tons of resources, they have great managers and great developers under their wing. Making a persistent identifier scheme for OSM data is no cat sneeze. They’re amazing and they have everything.

But they still have OpenStreetMap at their base. No matter how nice a deck you build, it’s what keeps the ship from sinking that matters. The hull. And here’s a reasonable question: with all these resources, what does Overture Maps plan to do to support the project at its core?

Nothing? That’s the answer from companies like Amazon, who have successfully forked Mongo, Postgres, Nginx and everything else and are improving those with their excellent engineers. Giving money doesn’t work either, as I previously wrote about in Shtosm. Because we see OSMF now, to quote “Fight Club”, at a very strange time in their lives. It seems like what I want from Overture is to “come in and fix things”. Help not so much with money, but with management, developers. Make the damn guild.

Right now they don’t come to OSM. “You guys are cool, keep it up”. OSMF isn’t going to Overture: “we’re open to communication, call us”. Two projects that will complement each other perfectly, that just need to approach each other. There are more than fifty people here at the conference connected to Overture, and even more from OSM. And they’re not talking! It’s not a technical disagreement, it’s not a legal disagreement, it’s not even attribution, as I explained to everybody yesterday. It’s a banal disconnect, a lack of understanding.

And that’s on Overture. They’re mostly made up of managers. People whose job it is to communicate with people. I don’t expect anything from OSMF, but the “suits” could’ve understood what’s going on here. But all they do is show how awesome their project is. Yeah, it’s awesome. But it’s not the right audience. This audience needs to be approached differently.

It’s a stalemate. I don’t know what to do. It’s a very strange moment for OpenStreetMap as a whole, and nobody’s talking about it.

On Help

What response would I expect from Marc to my question about what Overture Maps can do to help OpenStreetMap, other than thank yous?

It’s clearly not the money: we’ve seen that it doesn’t help. Not developers: as I wrote above, there are slightly different guys working on technology here than in corporations. Hosting conferences, as some of the channel joked? Well, that’s not something worth writing about.

Our project needs managers. We need a CEO. “CEO of OpenStreetMap” – how does that sound, huh? I can feel the heat: you promised to “help and guide”, not “sell and manage”! It’s not like that, that’s not what a manager is about, even at the highest level. Managers lead organizations in terms of tactics and strategy within a mission, just as product maintainers lead code projects. We need someone who can speak corporate.

Someone like Tyler. The former CEO of the Humanitarian Team was awesome. During the seven years with him, the NGO defined its mission, got a huge development grant for it, increased its annual budget 11 times, partnered with just about every other humanitarian organization, and created a network of hubs with over a hundred employees. Everyone knows HOT as an effective organization.

OSMF, on the other hand, can’t even launch microgrants, the most basic form of community incentives. Even less, the “achievement stars” I gave out in the form of OSM Awards ran out as soon as my job burned me out. The Board can’t work with neither the “top” nor the “bottom”, contacted only by people and organizations that don’t know it.

That’s fine: it seems like the Board is high above, but in reality, it’s just seven volunteers with technical backgrounds (mostly - techies have money and time for side work) who have their own jobs and hobbies. They are only expected to spend two or three hours a week on OSMF. It’s not about launching projects or business relations. It’s only about helping and responding. If it weren’t for Michelle and Dorothea, the Foundation would have decayed into nodes and relations a long time ago.

The top level managers make a legal entity out of an amateur batch, with which other organizations can communicate normally and which adequately represents its members. With a predictable budget, a strategy (the Board failed here, twice), behind-the-scenes contacts, an aura of successful success. The developer (a million cartographers in our case) makes the product, and the manager creates its future.

This is exactly what Overture Maps could help with at first. They have enough managers. They know whom to call. We don’t. There’s zero chance of us finding someone like Tyler. There’s no money for a first year CEO salary either. As I wrote above, it’s a stalemate and can only be resolved from the outside. That “outside” could be Overture — thus fixing several misunderstandings at once.

You might say, I’m dreaming. But I’m not the only one. It felt like this thought was circulating all around the conference on Sunday. I heard, even on the OSMF Board people agree on the necessity of management. But they don’t know how to express it, so they just sit and wait for proposals. OpenStreetMap may not have a future yet, but we already know roughly where to look for it.

Some Mastodon

Of course some discussion followed, and I also could not help ranting on a tangential topic, so here are some things from there.

• Ian Dees: “This was a big realization/growth for OpenStreetMap US. Hiring an Executive Director means more can happen and there’s consistent vision. Also, even one person’s full time is way more productive than several people’s volunteer-able time.”
• Brandon Liu asked, why I expect anything else of OSMF, when the status quo is good enough for everybody. To which I replied, “I’d start with “support”. I don’t feel supported when I map, it’s like stackoverflow in there. I don’t feel supported when I develop things, except for people using my apps and thanking me in person. I don’t feel supported when I document or propose things on wiki and around.”
• And: “Currently there isn’t any form of support, but a degree of opposition to anything. It takes a certain frame of mind to participate in OSM, especially in Europe.”
• The rant was about how OSM has been constantly losing best developers to corporations, and this continues on and on, because OpenStreetMap at its core is mappers, and only mappers matter.
• I pointed out that corporations push their resources into OSM QA: see daylight distribution, for example. And that’s weird: “imagine a company creating a validation toolchain for Wikipedia. How and why and what for? For OSM, these questions have clear and immediate answers.”
• And finally, “Actually I’ve just looked at OSMF mission statement and could not even find “support the community” line, just the servers. So that would be the change. For OSMF to start supporting OSM.”

Spent today writing a new Python library. Super useful if you are making command-line OSM processing scripts:

https://github.com/Zverik/cli-oauth2

With it you add OAuth2 authentication in just one line of code (well, 3-4 after PEP8).

auth = OpenStreetMapAuth(
    client_id, client_secret, ['read_prefs', 'write_api']
).auth_server(token_test=lambda s: s.get('user/details'))

user_name = auth.get('user/details.json').json()['user']['display_name']

This line starts a local web server, opens OSM OAuth page, catches the redirect, stores the token on disk, and returns a requests session that also prepends the API endpoint to its parameter.

Not very secure — but it doesn’t need to be. One drawback is when publishing sources to github, you would need to publish your client credentials as well. Or just read then from a config file, idk.

Already updated my Simple Revert and OSM to Sandbox scripts to use it. Hope it helps!

It’s that time again: we are ready to announce the third Baltics-located conference on OpenStreetMap and everything around it! Just like three years ago, we are joining the BalticGIT org team to have an opportunity to gather everyone interested in open data and open tools to present their work, meet other mappers and developers, and spend two days in a beautiful city on a super wide river.

Mark the date: 18-19 May (that’s Thursday and Friday!) in Riga, Latvia.

See the website for details.

The RIX airport is a home base to AirBaltic which has flights from like a hundred locations around the world. It’s really easy and affordable to come. So, we would be delighted to meet you there!

We are opening the call for papers. Please submit your topics, and we (me, that is) will contact you. We understand it’s just three months, but again, this is an OSM event, where there are new things every month :)

Registration will open a bit later, which we will also announce.

This week I’ve released the 2.0 version of Every Door, which irons out most of the inconveniences found within a month after its official release. Yes, the editor has been officially released, just a few days before two talks on it at SotM and FOSS4G (recordings pending). You should download the editor for your Android or iOS smartphone right now!

I absolutely love the experience it gives me. It has revived my love for plain mapping, going out and collecting things to put on the map. Pascal’s HDYC shows my mapping days went from 29 last year to over 120 this year. That’s because I again look around for unmapped things while outside, and making an edit no longer requires navigating a map on a small phone screen, or making scribbles to open JOSM later at home.

Looking at amenities mapped with Every Door, I feel that few mappers understand the power of the editor. My intention for it was to have most if not all shops and amenities mapped on OSM, to make our map better than proprietary alternatives not only wrt cycleways, but for everyday tasks like finding which flower shop is open at 8:30. This is a huge endeavour, as I see having mapped over a thousand shops in Tallinn and around Riga.

To do this, to have everything on the map, we need to adjust our mapping habits. “Record and map it later” would not work: later never happens. “Confirm this one attribute” does not work either: the map is missing everything, everywhere, always. We needed an ultimate tool to record as much as possible as fast as our fingers allow. Finally, there is one.

But to leverage its power, one must learn how it’s different, what to look for. Every Door is by no means simple. It’s not a novice-friendly editor like StreetComplete or iD: it needs getting used to. Training. Tutorials. I plan to maybe record a few videos on that. But today I’ll share the basics: how I use it on any day I see a shop or a restaurant.

On this picture you see the statistics for mappers and number of map changes made with Every Door in the last month. Estonia stands out: indeed, that’s where I live. Most of the changes are mine. But I’m not mapping every day: far from that. I’m not a hardcore mapper, I just like to add things to the map. I spend like couple hours a week on mapping. But there are two key things that enable me to collect data effectively.

First is time. There is always time. The bus comes in ten minutes? Great, let’s walk around the block and map building heights and entrances. Stuck at a playground with your child? Half a hundred micromapping edits for everything you see — easy. Have a spare hour returning from an event or waiting for somebody? Open the editor and walk into every shop and shopping centre along this main road.

And also dedication. Focus. When you’re mapping, it’s like learning a poem, or reading an article in an unfamiliar language. You do the same thing over and over, you look up new words and write them down. Mapping is getting to know your city over and over again. It’s learning it afresh. Every street and every building. Noticing and mapping every door.

You can see this on the picture again: that’s Overpass Turbo results for things I created or confirmed mostly on a single day, during my two hours of mapping. All offices in two office buildings, all shops around, everything in a shopping mall. Before Every Door, I just tried to remember or made a photo, to map at home. Photo mapping does not work. Mapillary mapping does not work. Your changes need to be done there and then. See something new? Open the editor, do a few clicks, and you’re amazing.

It’s simple, really: start with the “POI” mode, marked by a tea cup. Confirm or adjust your location with the map — do not zoom, just drag it if needed. And study the list of amenities below. Compare it with what you see around you. Spot the differences — and record them. Tap on cards to add opening hours or contact details. Tap (+) to add a missing shop. You can be sure what’s in the list is the 100% complete and correct representation of what’s in OpenStreetMap: if a shop is missing from the list, it’s missing from the map. Do add it.

Then it’s the same checklist: location, name, opening hours, wheelchair accessibility, payment options, optionally phone, website, operator. Should take you like twenty seconds. Save and proceed. Second, third, tenth shop. It took me 2.5 hours to map the second biggest shopping mall in Estonia the first time.

And it takes me under half an hour to walk it again to confirm everything. Confirmation is important. The check_date tag is important. Otherwise you cannot tell the probability of a shop still being there. Was it edited a week ago, or was it an automated edit for an object surveyed in 2010 and long gone?

In Every Door, when you see an amenity and you see that it’s mapped correctly and still there, you tap the checkmark. See these green checkmarks on the top picture? These are shops confirmed in the past two months. If a checkmark is black, and the shop is there, tap it. It’s a proper edit, it adds to your edits count and conveys important information to map users. When you’re not up to entering data, just confirm everything. It’s just a tap, and it’s as useful as answering a question in StreetComplete or clicking your way along a river in iD.

And that’s basically it: have the editor at hand, know how much time you’ve got, focus on mapping your city, use the POI list to validate amenities and fix the differences, and confirm POI that are mapped correctly.

I hope some of you would share my passion for mapping — not abstract mapping like collecting photos and not opening an editor, not automotive mapping, not tracing satellite imagery, but the mapping as it was done in early days of OpenStreetMap: just going out and adding everything you see with your hands onto the map. It benefits you, since you get to know corners of your city you didn’t know existed, and it benefits everybody else by having recent and correct info about all the shops and amenities in their Organic Maps or OsmAnd. Let’s get back that POI power from Google and Apple!

Ten years ago on this day we changed the license for our data to ODbL, Open Database License 1.0. That was the final action of the lengthy relicensing process, which followed an exciting show of redacting and remapping the planet.

In July 2012 we started every day with watching the redaction progress map, discussing how the redaction bot devoured our precious map data, and making memes on the way.

After the bot finished its work, we started remapping everything we had lost. Poland and Australia were particularly broken, but most other countries had their losses. Alas we could not make everybody agree to the new contributor’s terms, so some mapping had to be done twice. But the work went better than expected, and it was then when we felt that the community is more important than the map, and that OSM can survive the loss of the latter.

In mid-September, the first planet file under ODbL was published. Its date was 12th, which is today. Happy anniversary! I welcome you to watch the map from that time, marvel at how much we had mapped back then, and compare it to the current map:

https://2012.osmz.ru

You’re free to do whatever with the tiles, including downloading in bulk or stitching with BigMap 2 (there’s an “Enqueue” button that does the work for you). Tiles are published under the same license as the regular osm.org tiles. Please if you download tiles in bulk, do it responsibly, with delays. The server will go away on October 31st.

I thought for quite a few days on the Jochen’s study for the OSM data model, and I’ve no idea on whether it’s good or bad or how to improve it. The process feels pretty straightforward to me.

But that’s the issue. It’s not impossible. To fix OSM data issues, changing the data model is not important. Frankly, I don’t have anything to add to my 2019 talk at Heidelberg. It was a reply to Jochen’s and Andy’s musings on API 0.7 back then, and it is still now.

I know, Florence is too far away, and tickets are worth a fortune. I hesitated for weeks before shelling out 800 € for a flight and a hotel. That’s too much even for me, so I totally understand if you could not make it this year.

Thank gods for the Internet though! We are continuing with an option of virtual participation in SotM. Traditionally that have meant watching talks online and occasionally making people at the venue read your questions out loud. Not too fun when you have to concentrate on events happening thousands of kilometers away.

BUUUT this year we trying something different as well. First, passive participation: few people at the conference (or just me, we’ll see how it goes) would post photos and announcements and general impressions on everything going at and around the conference. It’s a way to get a glimpse at the experience of actually being there. I have done it in Russian for every SotM and FOSS4G and FOSDEM since 2016, and now I thought it might fit well into the virtual attendee package.

Also, active participation: record a video introduction of yourself and put it into the library of introductions. That way you can listen to people who attend the conference, either physically or virtually, and start a conversation with any of them. It is a magic tool that helps learn something about a person before approaching them: might help when you have an introvert’s anxiety (I should know!). And generally it’s fun: it highlights that a conference is not so much about talks and dinners, but about people first and foremost.

So, do buy a Venueless ticket at https://pretix.eu/osmf/SotM2022/ and join us: yes, you won’t be able to shake any hands or split a beer, but with a very little effort you can be a part of the conference.

After my last State of the Map talk, some asked me where’s that “Edit Tags” button on the osm.org website, to quickly fix any tags without launching Level0? Of course there wasn’t one: I just quickly made up a text area with Firefox developer tools. But the idea was there.

Now I’m proud to show you that the button works, with a series of changesets to prove it. Alas, not in the website itself: to enable it, you must install a browser extension. Get yours for Firefox or for Chrome. After installing, open the iD editor once, and then look at any object page on osm.org.

This extension is a hack. It uses some undocumented things and will break when something changes in the code. Like, you need to first open iD editor for the authentication to work. If you don’t see the “Edit Tags” link, refresh the page. It is flimsy, but you can edit the map with it.

Can this be done in the website code? Definitely, this change would be quite trivial. Couple hours tops. I’m tired of disputes with maintainers, but if you have a will to negotiate and push the pull request through the hoops — many people would thank you for that.

Today I’m proud to present my new OpenStreetMap editor. It’s called Every Door and works on both iPhones and Androids. I shared the idea last Summer at a State of the Map, but started writing just late October. In the last month and a half thirty people made ten thousand edits with the editor and helped make it much better. Now I’m launching the open testing.

The official website has links to TestFlight and Google Play, a short video, and a FAQ.

I’ve got just one feeling: at long last. One way or another I was suggesting something like that for OSM since 2013. Made a failed attempt with OpenSurveyor. Watched with hope for big company projects with paid developers — but all these have disappeared. In this time we’ve got one amazing StreetComplete, which I like a lot, although it’s not for me.

Every Door is not StreetComplete. It’s not for everybody who’s got a minute to answer a few questions on the phone. This editor is for dedicated mappers. For her who, walking in a mall with hundreds of shops, thinks it’d be awesome to add each one of these to the map. For him who, pushing a stroller, dreams of mapping every sandpit and swing. For “fanatics” ready to add every street lamp, bench, and every tree in a park. This is a proper editor, with which you can forget about photomapping and trace recording.

Install Every Door today and help me make it better. The open testing will commence till late June at least, and in that time we’ll update the interface, polish input fields, add many heuristics inside. And on the way we’ll map hundreds of thousands of shops. Anything to make using the editor fun and exciting, so that our map grows even richer.

There are multiple mapping tasks in OpenStreetMap that people still have to do old-style, like we did in 2010. But with current technology and with current funding of OSM research, these can be automated, improving mapping quality over the world.

If you work on OSM-related projects for a company, like in Facebook or Apple, please ask your manager if they can spare your time on projects useful for the community. For example:

1. Imagery alignment! People from the US and Western Europe won’t understand, but in most parts of the world, imagery is offset up to hundreds of meters. Mappers need to shift it to start mapping, otherwise different people would map with different offsets.

   We have plugins for automatically tracing roads, buildings and rivers. Why not create a button for JOSM and iD that looks at GPS traces (rasterized or not) and finds the best offset for an imagery layer?

2. okay I had just that one idea and struggle to come up with more

3. please share your ideas in comments

Reminding you that there is a big conference coming up: the State of the Map Baltics 2020 in Riga, Latvia. How big? Well, half a thousand attendees have already registered. It’s time for you to buy a flight and meet us and other GIS enthusiasts and professionals on 6th of March:

https://2020.sotm-baltics.org/

Why so big? Well, it’s just a part of a bigger conference: Baltic GIT is not related to version control systems, but to geospatial technologies. Think of it as a partly local FOSS4G, only with less focus on open source. And it’s free!

We’ve got Allan Mustard of OSMF, Stefan Gustafsson of ESA, Tomas Straupis, Kirill Bondarenko, Danil Kirsanov, Evgen Bodunov, Rihards Olups and dozens of other speakers and OSM members. Do visit, and let’s enjoy Riga together!

Note: this post goes too far and may be triggering. I believe OWG does great stuff, but is too powerfull with too few resources.

Thanks to a discussion in OSM Belarus telegram group, I suddenly realized why the new Board would not change anything. We could replace all seven members, and the project will stay the same. We could invite all GlobalLogic employees to run the project, and they wouldn’t harm a node. We are looking the wrong way.

All real work in OpenStreetMap is done by working groups. They decide on license terms, press relations, data policy, and technical stuff. Since we’ve got few volunteers, anybody can join any working group and… start working, I guess. The concept of “working” is what stops people from joining, and many people tried to explain there isn’t that much work.

In his manifesto Steve Coast suggests, besides closing off the Board and the tiles, to drastically increase budget for the Operations Working Group. Which makes sense: they are the only working group to have consistently spent money, which had measurable positive impact on the project. Why wouldn’t we want more servers?

OWG also manages our website. And planet extracts. And our data model. And developer relations. And has a final say over who can and who cannot integrate with our core systems. Which… Is a lot, don’t you think? They must have a vast experience on working in open source, on encouraging developers to join the effort.

Except they are doing exactly the opposite. Despite having 37 tile caching servers, our tiles are slower than ever, with many experienced contributors having moved to alternative tiles. In the past nine years I’ve seen dozens of developers driven off the Rails Port website code. Even active members of the Board were rejected their valuable contributions, like an endpoint to retrieve deleted objects. In their repositories, OWG members are consistently breaking every rule of open communities, by not giving outside people even a bit of control.

And who are the OWG? Strangely enough, in the past eight years OWG has only been shrinking. They are essentially just two people, who’s been in the project since 2006. I have no idea how they’ve managed to keep doing the same thing for 12 years, keeping the servers up 24/7, without any positive feedback.

Individually each of them are awesome, they have lots of experience, they know their job and their devops skills are sky high. They rush to fix our servers in the middle of night. We do not pay them: keeping volunteering to run a million-user project is no small feat. I doubt we can find other people like them. They are irreplaceable — and that is exactly why we need to stop relying on these people.

The Board can rotate on a yearly basis, fight against corporate involvements in matters that don’t matter, and spend five more years introducing microgrants that won’t affect the project in the slightest. Because the project is shut, because the real core group does not share power and shuns away anybody who thinks for a moment they can contribute. And even the hardware part is limited to reactive responses, since two volunteers do not have capacity to plan ahead.

With that, it seems like the only proactive and useful deed the new Board can do is to disband OWG completely.

They cannot go away themselves because of the burden of responsibility — a feeling I have shared myself in the past. Instead we need to plan our budget and hire people to do essential stuff. So they are responsible and humble, and do not have ambitions to run the project — only to make our hardware and devops better.

Managing our website and other core tools should be more open — and the level of OSM Carto is not enough, although it’s something to look up. We need a group of community managers, to attract and keep new developers, to be more bold with changes, to have yearly and quarterly plans for improving the user and corporate interactions. And to finally chart where the project is going.

As you might know, as of this week, Juno is no more. It is ceased to be. We had a good run, helped thousand of NYC drivers earn money and provided them with a human, attentive support team. But the increasing regulations in the city benefited Uber and disadvantaged their competitors, especially small companies like Juno. So we decided not to continue this fight.

What happens to the GPS tile layer we were making? It is still there — but not updating anymore. I have loaded two days worth of tracks for the central part, and month for the rest. It should help validate some of the complex cases around the edges, but obviously won’t tell you what has changed in recent days. Sorry for not updating it, and I hope it still helps with mapping New York.

In the meantime we have published another open-source thing: a reverse geocoder. In fact, it seems like the first ready to use, open and strictly reverse geocoder. Unlike others, it is in no way optimized for forward geocoding, but does one thing better than the others: finding an address for a coordinate. You can learn more about why it is not an easy task from my SotM lightning talk. And head to the github repository to see the queries and test cases for yourself:

https://github.com/gojuno/jrg

The moment Juno was over, the whole R&D team, me included, was acquired by Lyft. Which means, instead of asking others to open more data for OSM and to help the community with improving the map, I get to do that myself. Just for the US though, but still better than NYC only. I’m pretty excited for this new opportunity, and looking forward to working with their mapping team and their data — and making OpenStreetMap useful and better for everyone.

На прошлой неделе меня по работе отправили в Бобруйск. Это такой большой маленький город, где делают вкусный ирис, а с недавних пор мы в Juno Lab помогаем там делать умных школьников. Рассказывал, как обычно, про OpenStreetMap. Пришлось много тренироваться, чтобы дети смогли высидеть полтора часа и не заскучать. Разумеется, стоило им открыть iD, как их школа пропала с карты — но затем появилась, заново нарисованная.

Чтобы результат не пропал, повторил весь текст на камеру. Сорок минут отборной картографии. Теперь есть что присылать людям, которые не знают про OpenStreetMap, а вам лень объяснять:

https://www.youtube.com/watch?v=469pkgFDYhk

It all started with a bad edit. We in Juno rely on routing over OpenStreetMap roads, and notice every change that breaks the map matching. One day, we encountered a pretty big breakage caused by reversing some one-way roads. Turns out, all sources available to mappers, and even proprietary alternatives, like Google Street View, were obsolete enough to validate the edit. I told the story in my FOSDEM talk, but the main point is that it gave me an idea.

How fresh a source can be? We’ve got plenty of these, but they all are very old. GPS traces come very slowly, and people still trace lines from ten years ago. Satellite imagery is updated once every few years. Street View photos (that is, Mapillary) is always older than you’d like. If something, like a street direction, has been changed in past two months, you’ve got no way to know it and reflect it on the map, save for surveying it yourself. It’s okay for a small village, but not for New York.

We need NYC data to be as fresh as possible, so few weeks ago we have published all our GPS tracks for you to trace over. Not as raw data, of course, but as a raster tile layer, looking exactly the same as our standard GPS layer. But with more traces. Tens of thousands tracks — just for one day, never older than two days from now. It is updated daily, so remember to clear your cache when confused.

All the data comes from our drivers, so if you see a line on the layer, you can be sure somebody drove there yesterday or a day before. A line over highway=construction? Time to mark the road open. A line in a wrong direction over a one-way street? That’s an error in the map. A segment of highway=secondary with no lines over it? Check the news: it might have been closed recently. If you zoom close enough (tiles are rendered up to z19), you can spot turning lines, allowing for validating turn restrictions.

I believe this is the freshest geodata source available to OSM mappers. No other source consistently gives you data for updating a map as it was few days ago on this scale. The layer is local, but it can demonstrate a possible future for OSM sources.

Obviously, we take privacy seriously. All traces are trimmed from both ends by a random amount, and everything is cut by a bounding box, so you won’t find a lone track going to a specific neighbourhood of Trenton and back again. Time range is offset slightly, so there is no telling on which day specifically a ride occured. Finally, there is no metadata that could link any pixel from these tiles to our internal data or link to real people in some other way.

The technical part was completed in just a day thanks to the open source ecosystem of OpenStreetMap. A special thank you goes to Erica Fischer, who wrote GPX importing and visualizing scripts for OSM. If you are interested in how it works, it is quite simple: a script downloads all our GPX traces for a day, uses Erica’s scripts to make raster tiles, packages them all in a Docker container and uploads it to Amazon EC2.

The layer is already included in imagery suggestions in JOSM, when you edit inside New York City, and will be available in iD 2.15 when it’s released. For now you can use https://gps-tile.junolab.net/{zoom}/{x}/{y}.png for a custom layer (disable “Locator Overlay” to see the lines). The data is published under CC-BY-SA 4.0 with an explicit permission for tracing in OpenStreetMap. We are expecting to expand into New Jersey soon, but there are no fixed dates.

With this data, we aim not only to help mappers in the US to better map the NYC, but also to inspire employees of other companies to share their ride data. Publishing GPS tiles is safe, does not expose any secrets and helps people. If you work in Lyft, Uber, Yandex, Grab or other ride-sharing or car/scooter sharing service, please consider talking to your employer and publishing some open data. Contact me if you have questions or need help convincing.

“There are empty territories, but then a question arises: if they are completely empty, then why should they be mapped? For some reason, the local people decided that they don’t need to map that.”

“We, as white men, are very privileged, and we simply do not see obstacles.”

“Maybe in some form we will use the technologies made back in 2008-2009 for Osmarender”

“Why women can not program themselves an editor understandable to them, use tagging schemes that are necessary for women…“

“What this is about, is that Mapbox ruined the vector tiles for OSM.”

“To sum it up, we see that OSM is turning into Google Map Maker.”

Last week there was a big OpenStreetMap conference in Milan, where I obviously went. There were a lot of great talks, I met a hundred awesome people and shared many ideas. During the conference I wrote a live feed with thoughts and photos, though in Russian. That’s why I don’t plan to write a separate blog post: I’ve written enough.

At the end of the second day, when most people went to drink beer, me and four other attendees gathered to talk about OpenStreetMap and things learned at the conference. We recorded an 83 minute Russian-language podcast, which was shared on the live feed. But I also wanted to share it with the English-reading community, because some ideas from it were quite important, in my opinion.

Thus, I present to you a complete transcript of the podcast translated into English! I used Google Translate and spent couple hours on fixing words, so it should be understandable. I’d be happy if someone read it to the end and gave their thoughts on things discussed.

And I’d like to thank Dorothea again, who lent us a voice recorder. Without it, it would be unlikely the podcast could be deciphered at all.

This is a very rough translation of parts of the discussion that happened on the Shtosm Telegram channel. I’ve translated it so the upcoming vector tiles BoF section does not miss my point of view, while I’m presenting OSM Streak in another room.

Kate in her keynote mentioned the recent article by Richard about what OpenStreetMap needs:

http://blog.systemed.net/post/15

The answer is vector tiles, he writes, but before that he makes a long intro about OSM not being a map; it’s a community and possibilities. He is right. But if we talk about tiles, you should understand one thing. Developers of OSM Carto style have been discussing switching to vector since April. They approach it as a technical task: to repeat the same thing, but with a different stack. And Richard’s point is that it would be wrong.

Style developers are fixated on quality, on cartography, on target audience. The make a product: tiles, which are featured on thousands of web sites. And this product, being showcased on our website, is overshadowing the real product that we make: the geospatial database. That is bad, since people believe OSM is tiles and mapping style, not data. They expect from the website to show traffic, to feature a ruler and satellite imagery. People feel that our product is the website, the routing feature, the mapping style.

What we need is to push the style to the sidelines, and to employ vector tiles. Though not to repeat what we already have in them, because then we’d get exactly the same: a product that overshadows the data. We should do intentionally raw tiles, which adapt to user’s needs. Roads, buildings, some POIs by default. But with a few clicks you could see bicycle routes: make a photo and go cycling. A map like a clay: very plain, but take it — and the possibilities are endless.

This change would mean a revolution in how we relate to our data, especially from admins and style developers. It would require us to relieve control to the user. We do that with the data, but we are yet to learn to relieve control for other parts of the infrastructure. The time to learn is coming.

---

We’re discussing the openstreetmap.org website. I’m trying to convey the (not very fresh) idea that non-mappers do not need OSM, and don’t even need to know about the OSM. They can use other wonderful websites, a magnitude better in features and mapping style than our website. And that have a ruler. For example:

• http://maps.sputnik.ru/
• https://maps.mapcat.com/
• https://www.freemap.sk/
• maps.me application

If a person opened the OSM website, they came not for features and not for tiles, but to learn about OpenStreetMap. When we make a user-facing service with the website, then it’s weird that Tom closes issues about adding a ruler or improving directions. OSM website is not one of many useful tools for a user, but first of all, an instrument for mappers, and a showcase of OSM capabilities. And in that it is awful. For the price of being slightly useful for outside users — and in the end, nobody gains much.

So, the solution is to send outside people to outside websites, built with OpenStreetMap data, and starting to wreck our website into a fun place for mappers. As soon as OSM Carto is pushed back (not removed, just moved into secondary layers), and when we have a map builder on the front page, people would suddenly understand that all this times they were slaves to the single correct way of looking at OSM, and turns out they have options!

Improve the map builder tool a bit, so you can not only create, but also export maps — and you’re looking at a style database for hundreds of new styles. Then, add a style library, and we get styles for cyclists, styles with 100% POI displayed, pretty background styles for infographics, and thematic styles for those looking for a tobacco shop or a toilet, and weird coloured styles for modern art admirers. All that simply because OSM would stop building a third-tier google maps service, and will switch to pushing its unique qualities. For being simple and useful we have MapCat, and being that is not our thing. We can do better.

A new minor release of MAPS.ME is coming, and just now I’ve got the new data for it — with numbers on size and features difference with the old data. First, sizes:

• Indonesia_Central.mwm: 107 up to 127 MB
• Tanzania.mwm: 212 up to 225 MB
• US_Indiana_North.mwm: 19 up to 25 MB

Basically the same as the last time, except Portugal was replaced by Indiana state. And there was some prominent activity in all of Peru.

• Somebody has traced a few thousand highway=track around Buenos Aires.
• In Patagonia, Argentina, ~300 hamlets (50%) were demoted to isolated_dwellings.
• Austria_Carinthia: import of ~3000 address points, +50%.
• Brazil_Bahia: somebody mapped ~7000 power towers, +73%.
• Brazil_Rio Grande do Norte: from 5 salt ponds to 238.
• Canada_British Columbia_Islands: from 14 natural=cape to 635.
• Canada_Newfoundland_West: somebody almost doubled the number of mapped buildings: 487 → 806.
• Chile_North: 81% of place=hamlets disappeared.
• Congo-Kinshasa_Kivu: traced ~7000 segments of residential roads, +54%.
• The same in Gabon: +6000 segments, +160%.
• Ecuador_West: seems like many supermarkets were imported, 845→1923 (+128%).
• France_Pays de la Loire_Sarthe: somebody likes tracing fences, 350→3350.
• In French Alpes, especially in Ardeche, ~9000 intermittent streams were added.
• In Iceland, somebody retagged ~100 footways to cycleways, and added lit=yes tag to ~700 highway segments.
• India_Assam: +11500 buildings (+90%).
• India_Uttar Pradesh: 65 → 247 toilets.
• Indonesia: so many new roads mapped, most prominent +40k residential in Jawa Tengah and +32k tracks in central Indonesia.
• Ireland_Leinster: +250 wheelchair=limited tags (+54%).
• Japan_Chubu Region_Gifu: somebody had a great survey walk there, adding ~50 each shops, temples, information boards, graveyards, company offices etc.
• Mauritania: from 7 to 87 castles.
• Morocco: at least 12k new surface=* tags on roads.
• Namibia: all highway=road and highway=living_street tags were replaced.
• New Zealand: there’s an ongoing address points import, at least ~87k new (+65%).
• In Nothern Norway somebody imported (or mapped?) ~13k power towers.
• In Peru, schools and kindergartens were imported: +30k / +186% and +24k / +262%.
• In Poland, somebody is drawing area:highway=footway, at least 6k new objects.
• Russia_Arkhangelsk Oblast_North: almost all landuse=garages were removed, 135→18.
• Russia_Moscow: imported 2700 address points.
• Russia_Smolensk Oblast: somebody has a good mapping night, adding, among other stuff, 870 barrier=fences.
• Singapore: 158 → 84 landuse=military. Demilitarization or concealing?
• In South Africa many rural features were mapped, including farmlands, tracks and especially 2750 waterway=ditch lines.
• Spain_Catalonia_Provincia de Girona: 388 → 670 admin_level=4 boundaries.
• Spain_Andalusia_Sevilla and Spain_Cantabria: ~4000 new building:part objects, somebody making everything 3D?
• The same in Tanzania: +2000 building:part (+78%).
• UK_England_Greater London: from 78 down to 4 castles.
• UK_England_South West England_Bristol: deleted all but 99 riverbanks (-97%).
• US_California_Sacramento_Stockton: imported ~70k buildings, +56%.
• US_Colorado: removed 2/3 of address points, ~10k.
• US_Indiana_North: imported 150k address points, +600%.
• US_Texas: at least 16k new surface=* tags, doubling their count.
• Vietnam: 73 → 343 jewelry shops.

This list has taken more than a hour to write — but I imagine it took a million human-hours to map all that. Great work!

This is a translation of this article in Russian. Far from perfect, but understandable, I hope. Written by Ilya Zverev, CC-BY.

Riding the wave of Google Maps pricing news, Tom Chadwin reminded everyone of benefits of using open source solutions, and concluded with these words: “You now have a concrete compelling argument to those who have always asked: “Why not just use Google Maps?”.

Have I got a platinum argument towards a metaphorical Google Maps: because your open map does not have any future, that’s why. It doesn’t even have good POI coverage, unlike Google, which has franchise owners lined up with location offerings. Because it presents not a dozen grumpy dudes turning every data contributor down, but a nice mat with “Welcome” on it.

That’s an exaggeration, of course. We’ve got a great, beautiful map, which in many areas not only excels — it doesn’t have any alternatives. Nowhere else can you get a reasonably correct road graph. No other map would allow for estimating population density. Nobody would provide you with the data to install a copy of a service in a closed network.

With that, it is hard to not notice that OpenStreetMap is dying. Not because we’ve got a database for a map, or that we don’t have moderators, or the data is not split in layers, like Serge complained. For a technically skilled person it’s impossible to believe in the fall of OSM: the data is detached and decentralized, which is eternal by definition. On top of that, it is free (as in beer) and a million editors contribute to it: why isn’t every website using it?

Because it is unreasonable to use. OSM loses to any alternative maps for one reason: no control. Nobody has it. Over anything. Since around 2012 OpenStreetMap is headed directly into abyss, and rare attempts at correcting the course are met by grumpy blokes, who protect control levers, saying: “power to the community” and “our project self-regulates”. The project’s advantage became its weakness — and, it seems, the fatal one.

No control over the map. Want to import locations for your franchise? Tough luck, your data quality does not meet our standards. Want to map your town? Meet the local gatekeeper, who would scold you for highway tags choice and then disappear, because you are unbearable. And being a watchman is hard: in fourteen years the best we could do was OSMCha. Users of which are still complaining about wide, albeit thin, changesets. We’ve successfully lost the author of OWL. DWG members are still using outdated Perl scripts for work.

No control over the website. This is familiar to anybody who’ve made a pull request to any part of the core infrastructure. You’ll never hear a thanks, but will get a full bag of comments instead. Two guardians do not let through any unconventional change: it’s like amidst a crumbling world we must hold on to what we already have. They don’t see that the power of their grip is what crumbles their world.

No control over the data model. The last effort to advance the API required money and power of the entire Cloudmade company, which means a dozen osmers, working for venture funding for a few weeks. A hope for an “area” data type was faint five years ago, but by now even the most optimistic osmers stopped dreaming of change. The only thing in plans for the new API is restricting metadata for GDPR compliancy, and that’s only because nobody wants to pay charges.

No control over tagging. The main distinction and advantage of OpenStreetMap is a free tagging model. It has grew so enormous, nobody, not even experienced users, can choose correct tags. Forums are full of humor on heath, forest and namespaces. Proposals are a joke: one side invents alien tagging schemas of hundreds kilobytes, another turns inside out in attempts to sink every proposal. Novice mappers are not freaked out by this only because all editors, even on mobile, put tags away behind the presets.

No control over the mapping style. Long ago, the main mapnik style was so complex, people were afraid to touch it. Then it was converted to CartoCSS, made prettier, and contributors started flocking in. For a few years they were improving icons and colours, updated the database structure, sorted out fonts — and the map started looking decent, like on any other popular service. Same bleakness.

But now it’s obvious that nobody knows where to go next. Well, Paul Norman gives talks exactly about this (in my opinion) for two years. It’s painful to look at developers’ attempts to continue, especially this year: they fruitlessly try to change established tagging principles, because the OSM data model is incompatible with good carthography. We reached the ceiling of the rendering stack made five years ago. The only way out is to throw it all away and start anew — exactly what authors are discussing these days.

No control over developers. “The most precious resource is the community, which moves the project into its bright future.” Haha, just don’t look at developers, who move in any direction but forward. Some invent a twentieth geocoder, or tenth routing engine; some spend two weeks at data wrangling, only to come up with an unimpressive series of dots. “Hurray, I’ve managed to deploy a tile server”, we hear. Congratulations, but no. It’s 2018 outside, and we’ve got no developer environment, no integration tools, no financial support, no strategic plan. Just some lonely volunteers sitting on key elements of the infrastructure.

No control over license. Mappers want to protect their work, which is understandable. From this were born all the copyleft licenses, to make the world a better, more open place. But there was a flaw. To succeed in this world, you need to learn to bargain. To take a buildings dataset from government, and provide them with geometry updates. To allow a booking service to not open their data, as to get a million and a half verified hotels, and be able to improve position errors in a service used by hundreds millions. And so on.

Our license forbids all that, which doesn’t hurt third parties much: they’ve got enough data on their own. It is us who are hurt, because we cannot make any deals. Community members are on lookout to shut any advances. Even with trivial cases, we’ve got issues. Just this year I’ve seen half a dozen requests to use the map in various TV shows. And every time for similar questions, different answers were given. Nobody, not even the Legal Working Group, understands ODbL. But it is status quo, and in OpenStreetMap, status quo is king.

As you know, in this world to stay in place, you must run as fast as you can. I follow the news for Google Maps, Yandex Maps, 2GIS, and I see them trying new algorithms, fresh points of view. They change user interfaces, always improve data models, learn to communicate with their communities in new ways. They react to problems with structural changes. They have the power to change everything — or the opposite, to tidy up the data, smoothen edges, make it comfortable. They can buy and sell, to make their map better.

All the OpenStreetMap community can do is to gather for a weekend to trace buildings in yet another town. That’s why the main use cases for our project are humanitarian initiatives and serving as a base map layer, when one cannot afford a proper map. Try remembering, what major news we had in the past year, worthy of articles in big tech blogs? A new JOSM release with whitespace trimming in tags?

I agree that relying on a proprietary map means giving up some control to a corporation. But are you absolutely sure you want control over every part of carthographic stack? Do you have enough money for that? A commercial provider can change its terms and put you in an unconvenient situation, but unlike OSM, you can make deals with them. A company is people, who have all the power: you can call them and bargain for better limits, or ask for a help with mapping. You are a client to them; to OSM you, when wanting something for your business, are worse than nobody.

That’s why OpenStreetMap stopped growing. On the charts, you can notice negative trends brewing. Like Wikimapia around 2011, our project has tapped out its meanings. With the current direction, we have ten years, after which we would look like Wikimapia does now: lots of data, no community, who have defected to alternative projects. And then people who chose OSM as a replacement to Google Maps, would have to think again.

In the next year or two we must fix at least some of the issues and find new directions for the project. The “free as in beer” narrative, constant for the past ten years, turned from progressive to pathetic. The main question is, why would you want OpenStreetMap, when you can choose from any of alternative maps, each of which is better in some aspect. And don’t start with “but my yard is mapped better here”. Maybe we plan to revolutionize geography teaching worldwide, or become a universal base map for everyone, or become a framework for experiments in modern carthography. Any answer will do — as long as you are prepared to work.

Until then, most companies would prefer Google Maps.

Once or twice a month I build data for MAPS.ME application. After the process of downloading OSM data, building coastlines, extracting objects, generating indexes and packaging mwms is done, the final stage is testing. With that, I get notified when a file size changed dramatically, or many objects of a same type were added or removed in a region. I think you might be interested in some of these changes.

So, what happened between 16th March and 17th April?

• Indonesia_Central: file size increased from 67 to 107 MB
• Portugal_South: decreased from 56 down to 53 MB
• Tanzania: 183 up to 212 MB (and it was just 87 MB a year ago!)

And in objects of specific types:

• In Argentina somebody changed many road classes, mostly up (res→tertiary, primary→trunk).
• And number of place=town objects fell threefold.
• In Australia, somebody added surface=aspalt tag on a hundred thousand ways.
• Austria_Salzburg: 17 thousand more address points (24k → 41k).
• Bangladesh: office=ngo went from 76 to 121 objects.
• Bolivia_South: 123% increase in buildings (39k → 87k).
• Brazil around Minas Gerais: +30k power=tower and +300 substations.
• Brazil_South Region_West and Cape Verde: lost all highway=trunks (~50 ways each).
• Burundi: waterways increased by 100%, e.g. streams 6k → 12k objects.
• Canada_Ontario_Kingston: no more railway=tram.
• Czech_Stredni Cechy_East: post boxes: 450 → 1332.
• Germany_North Rhine-Westphalia_Regierungsbezirk Detmold: 64% of railway=light_rail converted to railway=tram.
• India_Andhra Pradesh: imported 8000 place=village/hamlet.
• India_Assam: mapped 6600 buildings (+108%).
• Indonesia_Central: mapped A LOT of roads (~40k tracks and unclassifieds) and 1.6 mln buildings (+180%).
• Italy_Marche: imported 3200 address points (+120%).
• Kenya: mapped 6k power=tower (+130%).
• Libya: mapped 9500 intermittent streams (up from 113).
• New Zealand North: imported 305k address points (+172%).
• Norway: imported many thousands of amenity=recycling all over the country, ~500% increase.
• Peru_Lima: shop=electronics count increased from 341 to 1447.
• Peru_South: mapped 43k buildings (+52%).
• Portugal_South: lost around half of natural=wood, =scrub, =grassland, landuse=farmland (36k to 20k), building:part (2500 to 800), railway=station (291 to 176).
• Russia_Saint Petersburg: leisure=stadium count went down from 172 to just 36. Yes, it’s one of FC 2018 cities.
• Rwanda and Sudan_West: lost all of highway=living_street.
• South Africa_Western Cape: added ~7k obscure amenity=* values.
• In Spain somebody retagged most amenity=doctors as amenity=clinic.
• Tanzania: building:part count went up from 1081 to 2655.
• In some regions of the United States, hundreds thousand highway=service ways appeared.