Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[error-tracking] Remove entry span fingerprinting limitation #26979

Merged
merged 3 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 3 additions & 43 deletions content/en/error_tracking/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ If you experience unexpected behavior with Error Tracking, the troubleshooting s

Datadog recommends regularly updating to the latest version of the Datadog tracing libraries, mobile SDKs, and web SDKs, as each release contains improvements and fixes.

## Errors are not found in Error Tracking
## Errors are not found in Error Tracking

### Logs

Expand All @@ -17,53 +17,15 @@ This [example query][3] searches for logs meeting the criteria for inclusion in
### APM

To be processed by Error Tracking, a span must have these attributes:

- `error.type`
- `error.message`
- `error.stack`

**Note**: The stack must have at least two lines and one *meaningful* frame (a frame with a function name and a filename in most languages).

Only errors from service entry spans (the uppermost service spans) are processed by Error Tracking. Error Tracking primarily captures unhandled exceptions, and this behavior is in place to avoid capturing errors handled internally by the service.

This [example query][5] searches for spans meeting the criteria for inclusion in Error Tracking.

#### Workarounds for bubbling up child span errors to service entry span

Some tracers provide a way to access the root span and bubble up the error from child to root.

{{< tabs >}}
{{% tab "Java" %}}

```java
final Span span = GlobalTracer.get().activeSpan();
if (span != null && (span instanceof MutableSpan)) {
MutableSpan localRootSpan = ((MutableSpan) span).getLocalRootSpan();
// do stuff with root span
localRootSpan.setTag("<TAG>", "<VALUE>");
}
```

{{% /tab %}}
{{% tab "Python" %}}

```python
context = tracer.get_call_context()
root_span = context.get_current_root_span()
root_span.set_tag('<TAG>', '<VALUE>')
```

{{% /tab %}}
{{% tab "Ruby" %}}

```ruby
current_root_span = Datadog.tracer.active_root_span
current_root_span.set_tag('<TAG>', '<VALUE>') unless current_root_span.nil?
```

{{% /tab %}}

{{< /tabs >}}

### RUM

Error Tracking only processes errors that are sent with the source set to `custom`, `source`, `report`, `network` or `console`, and contain a stack trace.
Expand All @@ -81,8 +43,6 @@ Spans associated with the error need to be retained with a custom retention filt
[1]: /help/
[2]: /logs/error_tracking/backend/?tab=serilog#attributes-for-error-tracking
[3]: https://app.datadoghq.com/logs?query=status%3A%28emergency%20OR%20alert%20OR%20critical%20OR%20error%29%20AND%20%28%40error.stack%3A%2A%20OR%20%40error.kind%3A%2A%29%20
[4]: /tracing/error_tracking/#use-span-tags-to-track-error-spans
[5]: https://app.datadoghq.com/apm/traces?query=%40_top_level%3A1%20%40error.stack%3A%2A%20AND%20%40error.message%3A%2A%20AND%20error.type%3A%2A%20AND%20%40_top_level%3A1%20
[5]: https://app.datadoghq.com/apm/traces?query=%40_top_level%3A1%20%40error.stack%3A%2A%20AND%20%40error.message%3A%2A%20AND%20error.type%3A%2A%20
[6]: https://app.datadoghq.com/rum/sessions?query=%40type%3Aerror%20%40error.stack%3A%2A
[7]: https://app.datadoghq.com/error-tracking/settings
[8]: /tracing/trace_collection/custom_instrumentation/java/dd-api/#set-tags--errors-on-a-root-span-from-a-child-span
2 changes: 1 addition & 1 deletion content/en/tracing/error_tracking/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ To get started with configuring your repository, see the [Source Code Integratio

## Use span attributes to track error spans

The Datadog tracers collect errors through integrations and the manual instrumentation of your backend services' source code. Error spans within a trace are processed by Error Tracking **if the error is located in a service entry span** (the uppermost service span). This span must also contain the `error.stack`, `error.message`, and `error.type` [span attributes][1] to be tracked.
The Datadog tracers collect errors through integrations and the manual instrumentation of your backend services' source code. Error spans within a trace are processed by Error Tracking if they are service top-level errors, whether or not they are located in service entry spans. This span must also contain the `error.stack`, `error.message`, and `error.type` [span attributes][1] to be tracked.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Datadog tracers collect errors through integrations and the manual instrumentation of your backend services' source code. Error spans within a trace are processed by Error Tracking if they are service top-level errors, whether or not they are located in service entry spans. This span must also contain the `error.stack`, `error.message`, and `error.type` [span attributes][1] to be tracked.
The Datadog tracers collect errors through integrations and the manual instrumentation of your backend services' source code. Error Tracking processes error spans within a trace if they occur at the service level, regardless of whether they are located in service entry spans. This span must also contain the `error.stack`, `error.message`, and `error.type` [span attributes][1] to be tracked.

I'm not 100% sure what is meant by "service top-level errors". Could you please help provide some clarification so we can make this clear? Thanks!


{{< img src="tracing/error_tracking/flamegraph_with_errors.png" alt="Flame graph with errors" style="width:100%;" >}}

Expand Down
Loading