[processor/logdedup] Use a specific field as deduplication key

[mauri870]

Component(s)

processor/logdedup

Is your feature request related to a problem? Please describe.

Currently the processor uses the entire log message as the deduplication key, allowing the user to remove certain fields from being considered during the hashing part via the exclude_fields option.

One scenario that is not covered is that the deduplication key could be already present as a body or attribute in the log message. You could exclude the other body and attributes via exclude_fields, but this negative match is very error prone and does not scale well.

Describe the solution you'd like

I propose we add a new option to use a certain LogRecord field as deduplication key. This option accepts ottl style path variables:

processors:
    logdedup:
        key:  body.my_id_field

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/logdedup: @MikeGoldsmith @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.