Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed handling of object attributes in flex output #2231

Merged
merged 3 commits into from
Aug 19, 2024

Conversation

joto
Copy link
Collaborator

@joto joto commented Aug 16, 2024

This is a backwards incompatible change in the way we are handling objects attributes (version, timestamp, changeset, uid, user) and the -x/--extra-attributes option in the flex output. The pgsql output is unaffected.

Old behaviour:

  • If -x is set: attributes are available in Lua
  • If -x is set: all objects are sent to the process functions, not only tagged objects
  • If -x is set: Attributes are stored in middle tables in slim mode

New behaviour:

  • Attributes are always available in Lua if they are in the data
  • Only tagged objects are sent to the process_node|way|relation functions. There are new functions process_untagged_node|way|relation that are always called for untagged objects if they are defined.
  • If -x is set: Attributes are stored in middle tables in slim mode (as before)

This separates out the three concerns (availability of attributes in Lua config, handling of untagged objects, and storage of attributes in middle).

Fixes #1680
Fixes #189

@joto joto force-pushed the new-attr-handling branch from dce55a4 to ed27806 Compare August 16, 2024 14:37
joto added 3 commits August 19, 2024 09:23
I did some benchmarking and it turns out there is real no measurable
difference between putting the data in there and not doing it. So
we'll just always do it. This means that it doesn't depend on the
-x/--extra-attributes command line option any more.

Note that the attributes will still be nil when coming from the
middle in update mode if -x/--extra-attributes was not used on import.
This happens regardless of whether the -x/--extra-attributes option is
set.
@joto joto force-pushed the new-attr-handling branch from ed27806 to daac353 Compare August 19, 2024 07:23
@lonvia lonvia merged commit 7968877 into osm2pgsql-dev:master Aug 19, 2024
27 checks passed
@joto joto deleted the new-attr-handling branch August 19, 2024 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split up --extra-attributes option Huge performance loss when using --extra-atributes
2 participants