From 5da3fb8899532928ba1950b789c89aed0c6b89ac Mon Sep 17 00:00:00 2001 From: Annosha Date: Mon, 21 Oct 2024 12:05:48 +0500 Subject: [PATCH 1/8] Update metrics customization guide with .NET 9 Advice API details --- docs/metrics/customizing-the-sdk/README.md | 55 ++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 8aaeef19fad..833a96e6e20 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -335,6 +335,61 @@ by using Views. See [Program.cs](./Program.cs) for a complete example. +### Explicit Bucket Histogram Aggregation with .NET 9 Advice API + +With the introduction of the .NET 9 Advice API, developers can +customize histogram metrics more effectively. The Advice API +allows for explicit bucket boundaries when defining histograms, +leading to more precise and informative metrics. + +#### Overview of the .NET 9 Advice API + +The .NET 9 Advice API provides enhancements for metrics collection, +enabling developers to specify how histograms should behave. +This feature is particularly useful when dealing with a known +distribution of values, as it allows for tailored metrics +collection that aligns with application requirements. + +#### Using Explicit Bucket Boundaries + +To use explicit bucket boundaries in your histograms, follow these steps: + +1. **Install Required Packages**: Ensure you have the latest version of the +OpenTelemetry .NET SDK that supports the Advice API. +2. **Define Histogram with Explicit Buckets**: When creating a histogram metric, +specify the bucket boundaries explicitly using the new API. + +**Example**: + +```csharp +using OpenTelemetry.Metrics; + +// Create a meter +var meter = new Meter("MyApplication"); + +// Define bucket boundaries +double[] bucketBoundaries = { 0.0, 10.0, 20.0, 30.0, 40.0, double.MaxValue }; + +// Create a histogram with explicit buckets +var histogram = meter.CreateHistogram("request_duration", "ms", "Measures the duration of requests", new HistogramOptions { BucketBoundaries = bucketBoundaries }); + +// Record values +histogram.Record(15.0); +histogram.Record(25.0); +``` + +#### Best Practices + +- **Select Meaningful Bucket Boundaries**: When defining bucket + boundaries, consider the typical range of values that your application + processes. This helps to ensure that the histogram provides relevant + insights. +- **Monitor and Adjust**: After implementing explicit bucket + boundaries, monitor the metrics collected and adjust the + boundaries as necessary to improve the accuracy and usefulness + of the histogram data. + + #### Change the ExemplarReservoir > [!NOTE] From 6f0f11b00b42140c71919085bb87bc4989eccad6 Mon Sep 17 00:00:00 2001 From: Annosha Date: Wed, 23 Oct 2024 13:48:53 +0500 Subject: [PATCH 2/8] Improvements based on feedback --- docs/metrics/customizing-the-sdk/README.md | 46 ++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 833a96e6e20..be04937c1aa 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -224,6 +224,52 @@ default boundaries. This requires the use of new ExplicitBucketHistogramConfiguration { Boundaries = Array.Empty() }) ``` +### Customizing OpenTelemetry .NET SDK for Metrics + +#### Using Explicit Buckets in Histograms + +In OpenTelemetry .NET, histograms can be customized using the View API, and the Advice API allows additional flexibility for defining explicit bucket boundaries in instrumentation. + + +##### View API vs. Advice API: Defining Histogram Buckets + +When both the View API and Advice API are used to define histogram bucket boundaries, + the following priority rules apply: + +1. **View API**: Always takes precedence if it explicitly specifies histogram buckets. +2. **Advice API**: If no custom buckets are specified via the View API, the + Advice API-defined buckets will be applied. +3. **SDK Defaults**: If neither the View API nor the Advice API specifies bucket + boundaries, the SDK will use its default configuration. + +Additionally, if an exponential histogram is specified via the View API, the + explicit bucket boundaries provided by the Advice API will be ignored, + as exponential histograms take precedence. + +##### Example: Using Explicit Buckets via the Advice API + +Here is an example of how to use the Advice API to specify explicit bucket +boundaries for a histogram: + +```csharp +using OpenTelemetry.Metrics; + +// Create a meter +var meter = new Meter("MyApplication"); + +// Define bucket boundaries +double[] bucketBoundaries = { 0.0, 10.0, 20.0, 30.0, 40.0, double.MaxValue }; + +// Create a histogram with explicit buckets using Advice API +var histogram = meter.CreateHistogram("request_duration", +"ms", "Measures the duration of requests", +new HistogramOptions { BucketBoundaries = bucketBoundaries }); + +// Record values +histogram.Record(15.0); +histogram.Record(25.0); +``` + ##### Base2 exponential bucket histogram aggregation By default, a Histogram is configured to use the From fe2816161c31ceef8254e2c105de945ced02d355 Mon Sep 17 00:00:00 2001 From: Annosha Date: Thu, 24 Oct 2024 17:13:12 +0500 Subject: [PATCH 3/8] Feedback-improvements --- docs/metrics/customizing-the-sdk/README.md | 39 +++++++++++----------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index be04937c1aa..012ae09e8d8 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -226,30 +226,33 @@ default boundaries. This requires the use of ### Customizing OpenTelemetry .NET SDK for Metrics -#### Using Explicit Buckets in Histograms +#### Determining Explicit Buckets for Histograms -In OpenTelemetry .NET, histograms can be customized using the View API, and the Advice API allows additional flexibility for defining explicit bucket boundaries in instrumentation. +In OpenTelemetry .NET, histograms can be customized using both +the View API and the Advice API. The actual explicit buckets used by +the SDK are determined based on the following factors: +1. The explicit buckets provided by the instrumentation owner via the Advice API. +2. The explicit buckets defined through the View API during SDK initialization. +3. The default buckets provided by the SDK. -##### View API vs. Advice API: Defining Histogram Buckets +Here’s how the SDK determines which explicit buckets to use: -When both the View API and Advice API are used to define histogram bucket boundaries, - the following priority rules apply: +1. **View API**: If custom buckets are defined via the View API, these +take precedence over any buckets provided by the Advice API or SDK defaults. +2. **Advice API**: If the View API does not specify custom buckets, the SDK +uses the explicit buckets provided by the Advice API. +3. **SDK Defaults**: If neither the View API nor the Advice API specifies +custom buckets, the SDK applies its default configuration. -1. **View API**: Always takes precedence if it explicitly specifies histogram buckets. -2. **Advice API**: If no custom buckets are specified via the View API, the - Advice API-defined buckets will be applied. -3. **SDK Defaults**: If neither the View API nor the Advice API specifies bucket - boundaries, the SDK will use its default configuration. - -Additionally, if an exponential histogram is specified via the View API, the - explicit bucket boundaries provided by the Advice API will be ignored, - as exponential histograms take precedence. +Additionally, if an exponential histogram is defined using the View API, +the explicit bucket boundaries provided by the Advice API will be ignored, +as exponential histograms take precedence. ##### Example: Using Explicit Buckets via the Advice API Here is an example of how to use the Advice API to specify explicit bucket -boundaries for a histogram: + boundaries for a histogram: ```csharp using OpenTelemetry.Metrics; @@ -260,10 +263,8 @@ var meter = new Meter("MyApplication"); // Define bucket boundaries double[] bucketBoundaries = { 0.0, 10.0, 20.0, 30.0, 40.0, double.MaxValue }; -// Create a histogram with explicit buckets using Advice API -var histogram = meter.CreateHistogram("request_duration", -"ms", "Measures the duration of requests", -new HistogramOptions { BucketBoundaries = bucketBoundaries }); +// Create a histogram with explicit buckets using the Advice API +var histogram = meter.CreateHistogram("request_duration", "ms", "Measures the duration of requests", new HistogramOptions { BucketBoundaries = bucketBoundaries }); // Record values histogram.Record(15.0); From 1e97051aea04396171b65641edc83919fc6def43 Mon Sep 17 00:00:00 2001 From: Annosha Date: Thu, 24 Oct 2024 18:59:16 +0500 Subject: [PATCH 4/8] linting fixes --- docs/metrics/customizing-the-sdk/README.md | 29 ++++++++++------------ 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 012ae09e8d8..6cb75275829 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -224,8 +224,6 @@ default boundaries. This requires the use of new ExplicitBucketHistogramConfiguration { Boundaries = Array.Empty() }) ``` -### Customizing OpenTelemetry .NET SDK for Metrics - #### Determining Explicit Buckets for Histograms In OpenTelemetry .NET, histograms can be customized using both @@ -436,7 +434,6 @@ histogram.Record(25.0); boundaries as necessary to improve the accuracy and usefulness of the histogram data. - #### Change the ExemplarReservoir > [!NOTE] @@ -581,11 +578,11 @@ recording of `Exemplar`s. OpenTelemetry SDK comes with the following `ExemplarFilter`s (defined on `ExemplarFilterType`): -* (Default behavior) `AlwaysOff`: Makes no measurements eligible for becoming an +- (Default behavior) `AlwaysOff`: Makes no measurements eligible for becoming an `Exemplar`. Using this disables `Exemplar` collection and avoids all performance costs associated with `Exemplar`s. -* `AlwaysOn`: Makes all measurements eligible for becoming an `Exemplar`. -* `TraceBased`: Makes those measurements eligible for becoming an `Exemplar` +- `AlwaysOn`: Makes all measurements eligible for becoming an `Exemplar`. +- `TraceBased`: Makes those measurements eligible for becoming an `Exemplar` which are recorded in the context of a sampled `Activity` (span). The `SetExemplarFilter` extension method on `MeterProviderBuilder` can be used @@ -618,9 +615,9 @@ environmental variables: Allowed values: -* `always_off`: Equivalent to `ExemplarFilterType.AlwaysOff` -* `always_on`: Equivalent to `ExemplarFilterType.AlwaysOn` -* `trace_based`: Equivalent to `ExemplarFilterType.TraceBased` +- `always_off`: Equivalent to `ExemplarFilterType.AlwaysOff` +- `always_on`: Equivalent to `ExemplarFilterType.AlwaysOn` +- `trace_based`: Equivalent to `ExemplarFilterType.TraceBased` #### ExemplarReservoir @@ -628,12 +625,12 @@ Allowed values: and is responsible for recording `Exemplar`s. The following are the default reservoirs: -* `AlignedHistogramBucketExemplarReservoir` is the default reservoir used for +- `AlignedHistogramBucketExemplarReservoir` is the default reservoir used for Histograms with buckets, and it stores at most one `Exemplar` per histogram bucket. The `Exemplar` stored is the last measurement recorded - i.e. any new measurement overwrites the previous one in that bucket. -* `SimpleFixedSizeExemplarReservoir` is the default reservoir used for all +- `SimpleFixedSizeExemplarReservoir` is the default reservoir used for all metrics except histograms with buckets. It has a fixed reservoir pool, and implements the equivalent of [naive reservoir](https://en.wikipedia.org/wiki/Reservoir_sampling). The reservoir pool @@ -667,12 +664,12 @@ Though `MetricReader` can be added by using the `AddReader` method on Refer to the individual exporter docs to learn how to use them: -* [Console](../../../src/OpenTelemetry.Exporter.Console/README.md) -* [In-memory](../../../src/OpenTelemetry.Exporter.InMemory/README.md) -* [OTLP](../../../src/OpenTelemetry.Exporter.OpenTelemetryProtocol/README.md) +- [Console](../../../src/OpenTelemetry.Exporter.Console/README.md) +- [In-memory](../../../src/OpenTelemetry.Exporter.InMemory/README.md) +- [OTLP](../../../src/OpenTelemetry.Exporter.OpenTelemetryProtocol/README.md) (OpenTelemetry Protocol) -* [Prometheus HttpListener](../../../src/OpenTelemetry.Exporter.Prometheus.HttpListener/README.md) -* [Prometheus AspNetCore](../../../src/OpenTelemetry.Exporter.Prometheus.AspNetCore/README.md) +- [Prometheus HttpListener](../../../src/OpenTelemetry.Exporter.Prometheus.HttpListener/README.md) +- [Prometheus AspNetCore](../../../src/OpenTelemetry.Exporter.Prometheus.AspNetCore/README.md) ### Resource From 76ad2b78ae177d62896a98483be3f597c878d34e Mon Sep 17 00:00:00 2001 From: Annosha Date: Thu, 24 Oct 2024 19:18:47 +0500 Subject: [PATCH 5/8] lint fixes --- docs/trace/customizing-the-sdk/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/trace/customizing-the-sdk/README.md b/docs/trace/customizing-the-sdk/README.md index d78ed1cd569..d8cade0be13 100644 --- a/docs/trace/customizing-the-sdk/README.md +++ b/docs/trace/customizing-the-sdk/README.md @@ -477,8 +477,7 @@ When using the `AddOpenTelemetry` & `WithTracing` extension methods the into an existing collection (typically the collection used is the one managed by the application host). The `TracerProviderBuilder` will be able to access all services registered into that collection. For lifecycle management, the -`AddOpenTelemetry` registers an [IHostedService -](https://learn.microsoft.com/dotnet/api/microsoft.extensions.hosting.ihostedservice) +`AddOpenTelemetry` registers an [IHostedService](https://learn.microsoft.com/dotnet/api/microsoft.extensions.hosting.ihostedservice) which is used to automatically start the `TracerProvider` when the host starts and the host will automatically shutdown and dispose the `TracerProvider` when it is shutdown. From f5472100bdd6833d32eec8457ab8d30728d3e1a6 Mon Sep 17 00:00:00 2001 From: Annosha <111076986+Annosha@users.noreply.github.com> Date: Tue, 29 Oct 2024 10:11:12 +0500 Subject: [PATCH 6/8] Update docs/metrics/customizing-the-sdk/README.md Co-authored-by: Reiley Yang --- docs/metrics/customizing-the-sdk/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 6cb75275829..174c04eef67 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -227,7 +227,7 @@ default boundaries. This requires the use of #### Determining Explicit Buckets for Histograms In OpenTelemetry .NET, histograms can be customized using both -the View API and the Advice API. The actual explicit buckets used by +the Advice API and the View API. The actual explicit buckets used by the SDK are determined based on the following factors: 1. The explicit buckets provided by the instrumentation owner via the Advice API. From 1d82d760489983b740ab395240f779bf73695f4f Mon Sep 17 00:00:00 2001 From: Annosha <111076986+Annosha@users.noreply.github.com> Date: Fri, 8 Nov 2024 21:37:19 +0500 Subject: [PATCH 7/8] Update docs/metrics/customizing-the-sdk/README.md Co-authored-by: Reiley Yang --- docs/metrics/customizing-the-sdk/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 07ed6c67093..f957925d863 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -250,7 +250,7 @@ as exponential histograms take precedence. ##### Example: Using Explicit Buckets via the Advice API Here is an example of how to use the Advice API to specify explicit bucket - boundaries for a histogram: +boundaries for a histogram: ```csharp using OpenTelemetry.Metrics; From aec0a7d1fd94ead74bb93b7301362d0aaa47a910 Mon Sep 17 00:00:00 2001 From: Annosha <111076986+Annosha@users.noreply.github.com> Date: Fri, 8 Nov 2024 21:37:33 +0500 Subject: [PATCH 8/8] Update docs/metrics/customizing-the-sdk/README.md Co-authored-by: Reiley Yang --- docs/metrics/customizing-the-sdk/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index f957925d863..a4a5b4b3643 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -234,7 +234,7 @@ the SDK are determined based on the following factors: 2. The explicit buckets defined through the View API during SDK initialization. 3. The default buckets provided by the SDK. -Here’s how the SDK determines which explicit buckets to use: +Here's how the SDK determines which explicit buckets to use: 1. **View API**: If custom buckets are defined via the View API, these take precedence over any buckets provided by the Advice API or SDK defaults.