Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Unix socket for internal telemetry metrics #11941

Open
avinovarov opened this issue Dec 17, 2024 · 2 comments
Open

Using Unix socket for internal telemetry metrics #11941

avinovarov opened this issue Dec 17, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@avinovarov
Copy link

avinovarov commented Dec 17, 2024

Describe the bug
Attempting to send internal telemetry over Unix socket fails, error is logged and the OTel process is terminated.

Steps to reproduce
In internal telemetry config, try to use unix:otlp.sock instead of URI in config.service.telemetry.metrics.readers[periodic].exporters.otlp.endpoint

What did you expect to see?
Unix socket recognized properly as internal telemetry endpoint, internal telemetry metrics are being sent to Unix socket in the same way app container in the same pod does.

What did you see instead?
Pod fails to start, sidecar OTel Collector container logs error:

2024/12/17 09:29:01 collector server run finished with error: failed to create metric provider: parse "http://unix:otlp.sock": invalid port ":otlp.sock" after host

What version did you use?
We run OTel Collectors in our k8s clusters, installed as Helm charts.
Chart version: opentelemetry-operator:0.74.3
OTel image version override: 0.115.1 (tried on 0.114.0 too)

What config did you use?

  config:
    service:
      telemetry:
        metrics:
          level: detailed
          readers:
            - periodic:
                interval: 10000
                exporter:
                  otlp:
                    protocol: grpc/protobuf
                    endpoint: unix:otlp.sock # <- only network URI is being accepted due to address:port validation

Environment
All workloads are run in Azure k8s/AKS v1.30

Additional context
It looks like the error hints on what's happening:

2024/12/17 09:29:01 collector server run finished with error: failed to create metric provider: parse "http://unix:otlp.sock": invalid port ":otlp.sock" after host

Instead of accepting the endpoint address the way it is, the validation adds http:// to the beginning of the string, and considers everything after colon as the port, like :<port_expected>.

OTel Collector OTLP exporter documentation refers to the gRPC name resolution specification which should allow unix sockets in unix:socket syntax, so this extra endpoint address validation looks excessive and actually breaks the usage of unix sockets.

@avinovarov avinovarov added the bug Something isn't working label Dec 17, 2024
@bogdandrutu
Copy link
Member

There is a bit of a problem here, the "exporter" under the periodic reader is not a collector exporter, but an otel-go exporter and I am not sure how they recommend that to be configured.

@mx-psi
Copy link
Member

mx-psi commented Dec 19, 2024

@open-telemetry/go-approvers is using Unix sockets supported by OTel Go?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants