Monday, July 5, 2010

Announcing the ArcGIS Editor for OpenStreetMap

For a while, ArcGIS users have been able to use the OpenStreetMap (OSM) content as a basemap in ArcGIS Desktop or in web applications thanks to a republishing of this content through ArcGIS Online. After the earthquakes, we have received many requests from users of ArcGIS who want to contribute to OSM, but who prefer to use the editing capabilities of ArcGIS Desktop.

For users of ArcGIS 10 this is now possible using the new free add-on ArcGIS Editor for OpenStreetMap.

The ArcGIS Editor for OpenStreetMap is designed to help ArcGIS desktop users to become an active member in the growing community of users building an open and freely available database of geographic data.

The provided tools allow the user to download data from the OSM servers and store it locally in a geodatabase. The user can then use the advanced editing environment of ArcGIS Desktop 10 to create, to modify, and to delete data. Once the edits are complete, the edit changes can be posted back to the OSM server and become available to all OSM users.

The interaction with the OSM server is accomplished using as set of geoprocessing tools to download, to manage, and to upload data.

A total of six tools support the a disconnected editing like workflow: download data from OSM, edit locally, and upload the result back to OSM.

OSM has a very flexible data model, to support some consistency in created feature types. However for more focused data capture activities, such as those that occurred after the Haiti and Chile earthquakes, a more focused data model approach is suggested. To use the new ArcGIS 10 template feature, we have mapped the common tags used in OSM to attributes and feature types, created templates for these, and implemented suggested symbols.

Editing is straightforward. After downloading your work area from OSM, you use the normal ArcGIS Desktop editing features. There are some things to keep in mind in this first release:
  • Only simple and single part geometries are supported.
  • You cannot create features with more than 5000 nodes. The OpenStreetMap server has a limit of accepting geometries with up to 2000 nodes.
  • Deleting a point, line, or polygon can have an effect of changing the relation in which the feature participates.
  • Editing of OSM relations or super-relations directly is not supported in this first release.
  • Polyons generated from data downloaded from OSM may be corrupt. To be safe: run the repair geometry tool before starting to edit.
As with any multi-user editing environment, you may run into a situation where multiple users edit the same area. This results in conflicts when trying to upload your edits to OSM. In order to mitigate the conflict the ArcGIS Editor for OpenStreetMap offers a simple Conflict Editor to help resolve the situation. Best practice is to edit a relatively small area and to save back to OSM frequently.

We are releasing this first version of the ArcGIS Editor for OpenStreetMap. and are looking for feedback. More details on the tool will become available over time, including access to the source code, and enhanced documentation.

Enjoy!

26 comments:

  1. Any chance we get the source code of those geoprocessing tools you have developed?

    ReplyDelete
  2. OSM has a 2000 node limit, not 5000...

    ReplyDelete
  3. "We are releasing this first version of the ArcGIS Editor for OpenStreetMap. and are looking for feedback. More details on the tool will become available over time, including access to the source code, and enhanced documentation. including access to the source code, and enhanced documentation."

    Great to hear that it will be open source! Thanks for this contribution.

    ReplyDelete
  4. Not being able to edit relations and, worse, potentially breaking them seems like a serious flaw.

    ReplyDelete
  5. While this is a great tool for downloading OSM data into a geodatabase for any given extent, I cannot get the edits I've made to upload to OSM. The script keeps returning a 404 (as in cannot find upload point) in the ArcToolbox Results tab. I second the motion to make this open source, as I'd be more than willing to poke around in an attempt to fix this issue.

    ReplyDelete
  6. By the way, I think there are some issues with authenticating against OSM. First, it seems to be unhappy with a trailing slash in the URL field of the Upload tool. Secondly, it doesn't seem to be completing the basic authentication against the web server, as it's not sending the Authorization header. I've only gleaned this much through packet sniffing. Hope this can be fixed soon (or I can be corrected if I'm doing something wrong) as this tool looks like it could be a great resource towards getting GIS users familiar with OSM.

    ReplyDelete
  7. @John: are you using http://api.openstreetmap.org/ to post your changes back to OSM? See the OSM Wiki for more info

    ReplyDelete
  8. @Chris: we'll make note of this as a feature enhancement. Right now we wanted to get the basic workflow and translation between OSM and ESRI information models under control.

    ReplyDelete
  9. @deltafoxtrot256: noted and updated the blog post.

    ReplyDelete
  10. @John it seems that the upload is a lot happier with an address that doesn't contain a trailing slash, so http://www.openstreetmap.org should work fine. I'll put it on the list of items to check and fix.

    ReplyDelete
  11. Marten and Thomas, it looks like I didn't try the combination of api.openstreetmap.org and omitting the trailing slash. That seems to have done it. I made some changes to Rowan University's campus using the tool, having reshaped, merged and deleted features. The changeset is 5157895.

    http://www.openstreetmap.org/browse/changeset/5157895

    Thanks again for making this tool!

    ReplyDelete
  12. I am having problems with the install. All it seems to do is install the documentation. There is no toolbox added to ArcMap.

    I am trying to install it on a WIN7 64bit box so I am not sure if thats the issue or not.

    Any ideas?

    Thanks,
    Matt

    ReplyDelete
  13. @Matt You are using ArcGIS 10, yes? in the folder where you installed OSMEditor, you should have a bin, data, and Documentation folder. in the data folder there should be a toolbox OpenStreetMap Toolbox.tbx. can you verify?

    ReplyDelete
  14. Thanks for responding to Matt on this one. I ran into the same issue & that's exactly the info that I needed. Why is that kind of basic info not included clearly in the documentation? Also, why is ESRI releasing this without any supporting documentation on their site?

    ReplyDelete
  15. @Mike: Point taken on documentation. We released this first version of the ArcGIS Editor for OpenStreetMap with the State of the Map Conference in mind happening this weekend in Spain. We are looking for feedback and your suggestion about documentation is on the mark.

    More details on the tool will become available over time (after the UC if you don't mind...), including access to the source code, and enhanced documentation.

    ReplyDelete
  16. Struggling with the OSM download. What projection/coordinate system does the bounding box request need to be in? I get 404s for pretty much everything I send, except for the occasional bbox too big message.

    ReplyDelete
  17. Yes I found it now i just need to figure out how to get data. I think I saw a post on that somewhere. Thanks for your help.

    Matt

    ReplyDelete
  18. all: The ArcGIS Editor for OpenStreetMap is now available as an open source add-on through codeplex: http://esriosmeditor.codeplex.com/ If you want to contribute to documentation, best practices, or code, send your codeplex account to mhogeweg at esri dot com and I can add you to the list of contributors.

    ReplyDelete
  19. Wondering if it would be possible for this to be compiled for ArcGIS 9.3.1... This would be awesome.

    ReplyDelete
  20. @Jeff It should be technically possible to do so but the editing experience would be quite different. Once you've used edit templates you never want to go back to the 'old' way of editing. In order to simulate or to approach the 10 edit templates you might need to adjust the feature property templates (xml documents) that are part of the editor extensions. Please let me know if you want to take on the compilation against 9.3.1 and I can go into a little more detail what needs to be adjusted.

    ReplyDelete
  21. "To use the new ArcGIS 10 template feature, we haveall: Have downloaded the install and mapped the common tags used in OSM to attributes and feature types, created templates for these, and implemented suggested symbols."

    successfully added the OSM toolbox tools and conflict editor toolbar. I cannot however seem to locate any OSM feature templates for editing, i.e. after starting an edit session my Create Feature template structure is empty.

    I cant seem to find any documentation on this. The screen shot on this article shows templates ready to use? Is these templates included with the download?

    ReplyDelete
  22. hi Trish, have you looked at the documentation on codeplex? http://esriosmeditor.codeplex.com/documentation Before starting an edit session, you will need to download some data from OSM. The templates will then be activated on the downloaded data.

    ReplyDelete
  23. Hi Marten, Thanks for the quick reply. I have read the documentation and am in the process of devising some workflows utilizing the OSM tools.
    I’ve accessed the templates using the "Download and Symbolize OSM" model. I found the best performance when running the tool in model mode (as opposed to the UI). The feature classes were downloaded and corresponding symbology were added to the TOC, however only line and polygon feature templates were loaded. To get access to the point feature templates I simply dragged the point layer file (included in the toolbar download), and remapped the data source . I was then able to access the point feature templates.
    I am now experimenting with the editing functionality by adding a few new amenities point features to the downtown Wellington area. I see that the input to the Upload OSM Data tool is a revision table however several of the field names, i.e. OSMCHANGESET, have me mystified. Should the revision table be updated as features are added/modified in the OSM dataset? Is there any documentation referring to this process?

    ReplyDelete
  24. Ahh ha - After you have your OMS data download and symbolized you must then enable OSM editing using the “Add OSM Editor Extension”. Once an edit session has been started, you can then modify any relevant OSM tag values (i.e. name and address information) through the attribute window.
    Once the OSM Editor Extension has been added, any changes made to you OSM data are tracked in the revision table. You can then upload your modified data using the Upload OSM Data tool, including this revision table as a required input. One note for new users – you must have an OSM account set up before being able to upload any changes.

    ReplyDelete
  25. @Trish - You got it! The documentation on codeplex (http://esriosmeditor.codeplex.com/wikipage?title=Use) has information on these steps. If you're up to it, I can grant you editing permissions and you can add your experiences to the docs.

    ReplyDelete