Micrometer OTLP

OpenTelemetry is a CNCF incubating project for providing standards for telemetry data. OpenTelemetry protocol (OTLP) is a vendor-neutral protocol that you can use to send data to various backends that support it. You can read the corresponding docs on how the metrics are ingested and can be visualized in the respective vendor docs.

1. Installing

For Gradle, add the following implementation:

implementation 'io.micrometer:micrometer-registry-otlp:latest.release'

For Maven, add the following dependency:

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-otlp</artifactId>
  <version>${micrometer.version}</version>
</dependency>

2. Configuring

The following example configures an OTLP registry:

OtlpConfig otlpConfig = new OtlpConfig() {
            @Override
            public String get(final String key) {
                return null;
            }
};

MeterRegistry registry = new OtlpMeterRegistry(otlpConfig, Clock.SYSTEM);

OtlpConfig is an interface with a set of default methods. If, in the implementation of get(String k), rather than returning null, you instead bind it to a property source (for example, a simple Map can work), you can override the default configuration through properties. For example, Micrometer’s Spring Boot support binds properties prefixed with management.otlp.metrics.export directly to the OtlpConfig:

management:
  otlp:
    metrics:
      export:
        # Supported configs
        url: "https://otlp.example.com:4318/v1/metrics"
        aggregationTemporality: "cumulative"
        headers:
          header1: value1
        resourceAttributes:
          key1: value1

  1. url - The URL to which data is reported. Defaults to localhost:4318/v1/metrics

  2. aggregationTemporality - Aggregation temporality determines how the additive quantities are expressed, in relation to time. The supported values are cumulative or delta. Defaults to cumulative.

  3. headers - Additional headers to send with exported metrics. This can be used for authorization headers. By default, headers are loaded from the config. If that is not set, they can be taken from the environment variables OTEL_EXPORTER_OTLP_HEADERS and OTEL_EXPORTER_OTLP_METRICS_HEADERS. If a header is set in both the environmental variables, the header in the latter overrides the former.

  4. resourceAttributes - Resource attributes are used for all metrics published. By default, Micrometer adds the following resource attributes:

Key Default value

telemetry.sdk.name

io.micrometer

telemetry.sdk.language

java

telemetry.sdk.version

<micrometer-core-version> (e.g.: 1.11.0)

service.name

unknown_service

If this config is empty, resource attributes are loaded from the OTEL_RESOURCE_ATTRIBUTES environmental variable. You can override service.name by setting the OTEL_SERVICE_NAME environmental variable, and this takes precedence over other configs.

3. Supported metrics

Metric points define the different data points that are supported in OTLP. Micrometer supports exporting the below data points in OTLP format,

  1. Sums

  2. Gauge

  3. Histogram

  4. Summary

The following table maps OTLP data points and the Micrometer meters:

OTLP data point Micrometer meter type

Sums

Counter, FunctionCounter

Gauge

Gauge, TimeGauge, MultiGauge

Histogram

Timer, DistributionSummary, LongTaskTimer, FunctionTimer (only sum and count are set)

Summary

Timer, DistributionSummary, LongTaskTimer

Note:

  1. max on the Histogram data point is supported only in delta aggregation temporality. This is because the values represented by cumulative min and max stabilize as more events are recorded and are less useful when recorded over application’s lifecycle.

  2. Currently, Micrometer only exports metadata for type Meter to OTLP.

4. Histograms and Percentiles

Micrometer Timer and DistributionSummary support configuring client-side percentiles and percentile histograms. The OTLP specification terms the Summary data point (client-side percentiles) as legacy and not recommended for new applications. The Summary data point also cannot have min/max associated with it. Due to these reasons, Micrometer prefers exporting Timers and DistributionSummary as a Histogram data point. By default, a Timer/DistributionSummary without any additional percentile/histogram config is exported as a Histogram data point. However, by configuring the timer to generate only client-side percentiles, by using publishPercentiles, you can change this to a Summary data point that exports pre-calculated percentiles. When both publishPercentiles and (publishPercentileHistogram or serviceLevelObjectives) are configured, the Histogram data point is preferred and pre-calculated percentiles are not generated. The following table describes which data point is used with different configurations:

Configuration OTLP data point

publishPercentiles

Summary

publishPercentileHistogram

Histogram

serviceLevelObjectives

Histogram

publishPercentiles and (publishPercentileHistogram/serviceLevelObjectives)

Histogram

Alternatively, if you use Spring Boot, you can use the per-meter properties to configure this behavior.

If you want to generate a Histogram data point for a Timer with name test.timer and default buckets generated by Micrometer, use:

management.metrics.distribution.percentiles-histogram.test.timer=true

For buckets with customized SLO, use:

management.metrics.distribution.slo.test.timer=10.0,100.0,500.0,1000.0

Alternatively, if you want to generate Summary data point for a timer with name test.timer and 90th and 99th percentiles, you can use:

management.metrics.distribution.percentiles.test.timer=0.9,0.99