promtail journal target should include "address field" timestamps (__REALTIME_TIMESTAMP, __MONOTONIC_TIMESTAMP)

[pbjhelmert]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The journal scrape target scrapes journal logs of all transports (syslog, stdout, etc). For the syslog transport, it's straightforward to parse the timestamps of such journal entries from the SYSLOG_TIMESTAMP field (among others), but stdout lines, there doesn't seem to be any timestamp field. For example, with this promtail configuration:

---
server:
  http_listen_port: 9080
  grpc_listen_port: 0
  log_level: debug

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
  - job_name: journal
    journal:
      json: true
      max_age: 12h
      labels:
        job: systemd-journal

And this promtail invocation:

# sudo ./promtail -config.file=config.yml -dry-run -inspect

We get syslog and stdout promtail output such as:

2025-01-03T12:45:32.811009-0600{__journal__boot_id="aa2f75fb6ca2418c954a3b89401c9620", __journal__gid="1000", __journal__hostname="localhost-live", __journal__machine_id="79d424b86c34404fa2c4930b25c66471", __journal__pid="15771", __journal__runtime_scope="system", __journal__selinux_context="unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023", __journal__source_realtime_timestamp="1735929932810712", __journal__transport="syslog", __journal__uid="1000", __journal_message="this is a test", __journal_priority="5", __journal_priority_keyword="notice", __journal_syslog_facility="1", __journal_syslog_identifier="jhelmert", __journal_syslog_timestamp="Jan  3 12:45:32 ", job="systemd-journal"}    {"MESSAGE":"this is a test","PRIORITY":"5","SYSLOG_FACILITY":"1","SYSLOG_IDENTIFIER":"jhelmert","SYSLOG_TIMESTAMP":"Jan  3 12:45:32 ","_BOOT_ID":"aa2f75fb6ca2418c954a3b89401c9620","_GID":"1000","_HOSTNAME":"localhost-live","_MACHINE_ID":"79d424b86c34404fa2c4930b25c66471","_PID":"15771","_RUNTIME_SCOPE":"system","_SELINUX_CONTEXT":"unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023","_SOURCE_REALTIME_TIMESTAMP":"1735929932810712","_TRANSPORT":"syslog","_UID":"1000"}

2025-01-03T12:46:53.539801-0600{__journal__boot_id="aa2f75fb6ca2418c954a3b89401c9620", __journal__cap_effective="1ffffffffff", __journal__comm="echo", __journal__gid="0", __journal__hostname="localhost-live", __journal__machine_id="79d424b86c34404fa2c4930b25c66471", __journal__pid="15877", __journal__runtime_scope="system", __journal__selinux_context="system_u:system_r:init_t:s0", __journal__stream_id="27328b5e44d340dbb4fdc9044609dadf", __journal__systemd_cgroup="/system.slice/test.service", __journal__systemd_invocation_id="951e8a8f72e44239966c347933e0d0ec", __journal__systemd_slice="system.slice", __journal__systemd_unit="test.service", __journal__transport="stdout", __journal__uid="0", __journal_message="this is a stdout log message", __journal_priority="6", __journal_priority_keyword="info", __journal_syslog_facility="3", __journal_syslog_identifier="echo", job="systemd-journal"}       {"MESSAGE":"this is a stdout log message","PRIORITY":"6","SYSLOG_FACILITY":"3","SYSLOG_IDENTIFIER":"echo","_BOOT_ID":"aa2f75fb6ca2418c954a3b89401c9620","_CAP_EFFECTIVE":"1ffffffffff","_COMM":"echo","_GID":"0","_HOSTNAME":"localhost-live","_MACHINE_ID":"79d424b86c34404fa2c4930b25c66471","_PID":"15877","_RUNTIME_SCOPE":"system","_SELINUX_CONTEXT":"system_u:system_r:init_t:s0","_STREAM_ID":"27328b5e44d340dbb4fdc9044609dadf","_SYSTEMD_CGROUP":"/system.slice/test.service","_SYSTEMD_INVOCATION_ID":"951e8a8f72e44239966c347933e0d0ec","_SYSTEMD_SLICE":"system.slice","_SYSTEMD_UNIT":"test.service","_TRANSPORT":"stdout","_UID":"0"}

For comparison, here's the respective JSON for those logs straight from journalctl:

{
        "MESSAGE" : "this is a test",
        "__CURSOR" : "s=8dc9f7fdf345436289f0ed614172bb73;i=1c08;b=aa2f75fb6ca2418c954a3b89401c9620;m=1afe65be1;t=62ad1af000b01;x=c77ed3effa834f23",
        "_MACHINE_ID" : "79d424b86c34404fa2c4930b25c66471",
        "_GID" : "1000",
        "_SELINUX_CONTEXT" : "unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023",
        "_TRANSPORT" : "syslog",
        "_HOSTNAME" : "localhost-live",
        "SYSLOG_FACILITY" : "1",
        "_PID" : "15771",
        "SYSLOG_TIMESTAMP" : "Jan  3 12:45:32 ",
        "_BOOT_ID" : "aa2f75fb6ca2418c954a3b89401c9620",
        "__SEQNUM" : "7176",
        "__SEQNUM_ID" : "8dc9f7fdf345436289f0ed614172bb73",
        "_SOURCE_REALTIME_TIMESTAMP" : "1735929932810712",
        "__REALTIME_TIMESTAMP" : "1735929932811009",
        "_UID" : "1000",
        "PRIORITY" : "5",
        "SYSLOG_IDENTIFIER" : "jhelmert",
        "__MONOTONIC_TIMESTAMP" : "7246076897",
        "_RUNTIME_SCOPE" : "system"
}

{
        "_COMM" : "echo",
        "_PID" : "15877",
        "_RUNTIME_SCOPE" : "system",
        "SYSLOG_FACILITY" : "3",
        "__MONOTONIC_TIMESTAMP" : "7326805688",
        "__SEQNUM" : "7278",
        "_MACHINE_ID" : "79d424b86c34404fa2c4930b25c66471",
        "PRIORITY" : "6",
        "_GID" : "0",
        "_SELINUX_CONTEXT" : "system_u:system_r:init_t:s0",
        "SYSLOG_IDENTIFIER" : "echo",
        "__SEQNUM_ID" : "8dc9f7fdf345436289f0ed614172bb73",
        "_SYSTEMD_CGROUP" : "/system.slice/test.service",
        "_STREAM_ID" : "27328b5e44d340dbb4fdc9044609dadf",
        "__REALTIME_TIMESTAMP" : "1735930013539801",
        "_SYSTEMD_UNIT" : "test.service",
        "_TRANSPORT" : "stdout",
        "_CAP_EFFECTIVE" : "1ffffffffff",
        "_SYSTEMD_SLICE" : "system.slice",
        "_UID" : "0",
        "__CURSOR" : "s=8dc9f7fdf345436289f0ed614172bb73;i=1c6e;b=aa2f75fb6ca2418c954a3b89401c9620;m=1b4b62eb8;t=62ad1b3cfddd9;x=bb234bf35d3fe43e",
        "_BOOT_ID" : "aa2f75fb6ca2418c954a3b89401c9620",
        "MESSAGE" : "this is a stdout log message",
        "_HOSTNAME" : "localhost-live",
        "_SYSTEMD_INVOCATION_ID" : "951e8a8f72e44239966c347933e0d0ec"
}

Describe the solution you'd like
A clear and concise description of what you want to happen.

By my eye, stdout transported journal entries only have timestamps of __REALTIME_TIMESTAMP and __MONOTONIC_TIMESTAMP, but these seem to be excluded from promtail's output. Since these seem to be the only timestamps included with all journal entries, I think it would be useful to include these fields to simplify the parsing of journal output (and accommodate stdout entries).

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

I don't see any alternative way to get timestamps for such journal entries if these fields are all the underlying journal data includes.

[rgroothuijsen]

The journal-scraping package used by promtail doesn't include the address fields in the fields set, but extracts them as metadata instead. The realtime timestamp is then used as the timestamp for the Loki entry, so it's included indirectly.