Update mappings for OTEL jvm metrics

[lenin-jaganathan]

opentelemetry-mapping-go/pkg/otlp/metrics/runtime_metric_mappings.go

Lines 92 to 224 in 5bac6ca

	var javaRuntimeMetricsMappings = runtimeMetricMappingList{
	"process.runtime.jvm.threads.count": {{mappedName: "jvm.thread_count"}},
	"process.runtime.jvm.classes.current_loaded": {{mappedName: "jvm.loaded_classes"}},
	"process.runtime.jvm.system.cpu.utilization": {{mappedName: "jvm.cpu_load.system"}},
	"process.runtime.jvm.cpu.utilization": {{mappedName: "jvm.cpu_load.process"}},
	"process.runtime.jvm.memory.usage": {{
	mappedName: "jvm.heap_memory",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.non_heap_memory",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"non_heap"},
	}},
	}, {
	mappedName: "jvm.gc.old_gen_size",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"G1 Old Gen", "Tenured Gen", "PS Old Gen"},
	}, {
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.gc.eden_size",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"G1 Eden Space", "Eden Space", "Par Eden Space", "PS Eden Space"},
	}, {
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.gc.survivor_size",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"G1 Survivor Space", "Survivor Space", "Par Survivor Space", "PS Survivor Space"},
	}, {
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.gc.metaspace_size",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"Metaspace"},
	}, {
	key: "type",
	values: []string{"non_heap"},
	}},
	}},
	"process.runtime.jvm.memory.committed": {{
	mappedName: "jvm.heap_memory_committed",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.non_heap_memory_committed",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"non_heap"},
	}},
	}},
	"process.runtime.jvm.memory.init": {{
	mappedName: "jvm.heap_memory_init",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.non_heap_memory_init",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"non_heap"},
	}},
	}},
	"process.runtime.jvm.memory.limit": {{
	mappedName: "jvm.heap_memory_max",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"heap"},
	}},
	}, {
	mappedName: "jvm.non_heap_memory_max",
	attributes: []runtimeMetricAttribute{{
	key: "type",
	values: []string{"non_heap"},
	}},
	}},
	"process.runtime.jvm.buffer.usage": {{
	mappedName: "jvm.buffer_pool.direct.used",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"direct"},
	}},
	}, {
	mappedName: "jvm.buffer_pool.mapped.used",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"mapped"},
	}},
	}},
	"process.runtime.jvm.buffer.count": {{
	mappedName: "jvm.buffer_pool.direct.count",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"direct"},
	}},
	}, {
	mappedName: "jvm.buffer_pool.mapped.count",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"mapped"},
	}},
	}},
	"process.runtime.jvm.buffer.limit": {{
	mappedName: "jvm.buffer_pool.direct.limit",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"direct"},
	}},
	}, {
	mappedName: "jvm.buffer_pool.mapped.limit",
	attributes: []runtimeMetricAttribute{{
	key: "pool",
	values: []string{"mapped"},
	}},
	}},
	}

OTEL Semantic conventions have renamed a lot of JVM metrics and lot of them were made stable as well. DataDog's translator still seem to use the old conventions and applications on new version of OTEL Semconv are missing the built-in dashboards due to this.

[lenin-jaganathan]

On a side note, renaming the OTEL semantic convention metrics to datadog-specific attributes seems to cause a lot of confusion among the users.
Is there any discussion to not rename these metrics but still keep the capabilities like built-in dashboards that can support both DD Agent generated metrics and OTEL Semantic convention metrics? This will at least keep the original metric names as is and will help users plotting these metrics.

[lenin-jaganathan]

The mappings are updated via #265.

But I would like to get an answer for

Is there any discussion to not rename these metrics but still keep the capabilities like built-in dashboards that can support both DD Agent generated metrics and OTEL Semantic convention metrics?

[mackjmr]

@lenin-jaganathan Thanks for providing the feedback regarding the metrics mapping. Addressing this is currently on our roadmap, and we'll make sure to reach back out once we have more information to share.

[lenin-jaganathan]

Thank you. One additional query is that the translations for JVM metrics covers memory/threads/classes but not the cpu/gc part of things. I know this might be a bit out of scope for this issue but just wanted to know is there anything I can read about this?

[liustanley]

@lenin-jaganathan Our translations for JVM metrics also include cpu/gc metrics. Some Datadog metrics such as jvm.gc.major_collection_count are missing because there is no corresponding OTel metric yet. Our documentation has more details on which metrics we provide translations for: https://docs.datadoghq.com/opentelemetry/integrations/runtime_metrics/java/.

[lenin-jaganathan]

@liustanley Thanks for that. Sorry, I missed the CPU metrics that are getting translated. Probably overlooked since they don't show up in the runtime metrics section in the Service overview page of DD.

Coming to the GC metrics, OTEL Semantic Convention emits jvm.gc.duration. This has the gc information which can be used to get the major/minor collection (for generational GC's) times as well as counts. Just need the mappings to be updated

e.g jvm.gc.duration{jvm.gc.name="G1 Young Generation"}

[lenin-jaganathan]

@liustanley
Something similar to this can be added to the OTEL runtime mapping as well.