Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid intermediate MapDB or PBF file when building network #31

Open
abyrd opened this issue Aug 4, 2017 · 0 comments
Open

Avoid intermediate MapDB or PBF file when building network #31

abyrd opened this issue Aug 4, 2017 · 0 comments

Comments

@abyrd
Copy link
Member

abyrd commented Aug 4, 2017

We're often pulling our OSM data out of VEX which will soon be backed by an RDBMS. That means the workflow will be RDBMS -> PBF -> MapDB -> R5. We could instead just do RDBMS -> R5.

It is tempting to just push the data from the RDBMS side through to the network builder in nodes, ways, relations order, but there are a few problems with that. The network builder can't just consume OSM entities in the order nodes, ways, relations because it may need to go back and look at the characteristics of certain nodes while it's building the edges. We don't even use the tags on the nodes, just their locations which are stored in the resulting vertices. But of course we only create vertices for the intersection nodes. And we need to iterate over all the ways once in order to find out which nodes are intersections.

So we'd have to push all the OSM data in a bounding box through several times, skipping huge chunks of the data each time. That's not more efficient. So instead we need to pull the data from the network builder side. First iterate over all the ways to find intersections or issue a query that just returns all the intersection nodes. Then fetch all the ways, and sub-fetch all the constituent nodes for each one.

Optimizations: switch off initializing the tags on the nodes (actually, very few nodes have tags so maybe not). Filter to return only highway ways and restriction relations, after filtering down to a box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant