New version for big OSM files?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

New version for big OSM files?

aruizga7
Hello!

I would like to know when the new version of osm2network will be ready. It is supposed that BIG osm files will be able to be converted as well right? Because I have 4 gb memory and I am always out of memory when I try to generate the network for big files...

Thank you in advance and congratulations because you are doing a great job!
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

peterseva
Hello!

Thank you for the congratulations!

We haven't planned a specific release date for the new version for big OSM files. At the moment I don't have much time as I'm currently working on my master's thesis.

Have you already tried to extend the heap size?

http://52north.org/communities/geoprocessing/osm2nds/installation.html

Best regards,
Eva
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

aruizga7
Hi!

Yes I already tryed. I will try to find a computer with more memory then.

Thanks and good luck with the thesis (I am also finishing my master thesis ;) )

Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

thughes
Hi folks... Is there a .osm file size limit for conversion? I am trying to convert a 413 Mb file for New South Wales, Australia and it appears to be hanging at the 'read' stage. I have increased the Heap size to 1024 so perhaps I need to be more patient! Has anyone successfully converted a file of this size?

I hope to get this working as it really is a great tool - thanks Eva!
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

peterseva
Hey!

This questions is often asked. Yes, the size depends on the memory. We are currently working at a new version for huge OSM files. This has a high priority.

If you don't need all information of your OSM file (small streets, some nodes and ways), you could extract only the most important parts with osmosis.

Best wishes,
Eva
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

ccriollo
Hi Everyone!

I am using this super useful tool to build networks as well :)

Based on your experience what is the biggest size for the OSM file?

I dont want to split my model...

Any advise?
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

peterseva
Hello Cesar,

the biggest region I have ever converted was the Canton of Zurich.

Some other users have experiences with big OSM files, too.

We would like to adapt the tool for OSM files of any size. Unfortunately, I currently don't have much time for this.

Best wishes,
Eva
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

Thomas
In reply to this post by ccriollo
We currently have an in-house version that will work against "any" size. So far I have only tried the state of California which is roughly half the size of Germany (with respect to the OSM content).
The current plan is to have a beta1 version of the ArcGIS Editor for OpenStreetMap 2.1 available shortly after the User Conference. Please watch http://esriurl.com/osmeditor for updates and announcements.

Thanks,
- Thomas
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

ccriollo
Hi Eva and Thomas!

Im in a bit of a rush, because I need to build 3 networks, Budapest, Munich and Bogota, 3 times pedestrian, cycling and vehicular.

The last two are massive, and they crash OSM2NDS even in 8gb W7 machine...

Can I try any beta of the OSMeditor Thomas? or Is there a work around to reduce the size of the OSM files? I meant, removing buildings and all features not required for building the network, for instande buildings and so on?

I really appreciate any advise, time is against me now...!

Thanks!

PS. if you need any feedback on testing and similar let me know :)
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

peterseva
Hello Cesar,

there is actually a workaround. You can use osmosis (http://wiki.openstreetmap.org/wiki/Osmosis) to remove everything except for:

highways (ways with the tag highway=*) and their referenced nodes
barriers (nodes with the tag barrier=*)
turn restrictions (relations with the tags type=restriction and restriction=*) with their referenced ways and nodes

Have a look at:

http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage

Best wishes,
Eva
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

ccriollo
Hallo Eva!

Thanks for the tip was useful!

Yet there is another addition, you can use QuantumGis with the OSM plugin for taking off the buildings or closed polygons from the OSM file, that was way easier than Osmosis.

And then Osmosis for clipping the right area! that is super fast :)

Just a piece of advise, run everything on Windows XP, I had tons of problems running anything on Windows 7, some of them no idea how to solve them...

Finally, OSM2NDS works on a 3gb WinXp with a maximum of OSM-200mb file! and I got all network data sets done!

Thanks!

PS. later will try to deploy the ArcGis OSM editor, I need another machine and ArcGis 10.1... :S
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

peterseva
Hello Cesar,

that's great!

Just ask if you have any further questions.

Best wishes,
Eva
Reply | Threaded
Open this post in threaded view
|

Re: New version for big OSM files?

tomford
Ho wow, I have to try it. Thanks for the tip. Thanks