Using Micrometer Tracing Directly

In this section, we describe how to use the Micrometer Tracing API to directly create and report spans.

Micrometer Tracing Examples

The following example shows the basic operations on a span. Read the comments in the snippet for details:

// Create a span. If there was a span present in this thread it will become
// the `newSpan`'s parent.
Span newSpan = this.tracer.nextSpan().name("calculateTax");
// Start a span and put it in scope. Putting in scope means putting the span
// in thread local
// and, if configured, adjust the MDC to contain tracing information
try (Tracer.SpanInScope ws = this.tracer.withSpan(newSpan.start())) {
    // ...
    // You can tag a span - put a key value pair on it for better debugging
    newSpan.tag("taxValue", taxValue);
    // ...
    // You can log an event on a span - an event is an annotated timestamp
    newSpan.event("taxCalculated");
}
finally {
    // Once done remember to end the span. This will allow collecting
    // the span to send it to a distributed tracing system e.g. Zipkin
    newSpan.end();
}

The following example shows how to continue a span in a new thread that was started in another thread:

Span spanFromThreadX = this.tracer.nextSpan().name("calculateTax");
try (Tracer.SpanInScope ws = this.tracer.withSpan(spanFromThreadX.start())) {
    executorService.submit(() -> {
        // Pass the span from thread X
        Span continuedSpan = spanFromThreadX;
        // ...
        // You can tag a span
        continuedSpan.tag("taxValue", taxValue);
        // ...
        // You can log an event on a span
        continuedSpan.event("taxCalculated");
    }).get();
}
finally {
    spanFromThreadX.end();
}

The following example shows how to create a child span when explicitly knowing who the parent span is:

// let's assume that we're in a thread Y and we've received
// the `initialSpan` from thread X. `initialSpan` will be the parent
// of the `newSpan`
Span newSpan = this.tracer.nextSpan(initialSpan).name("calculateCommission");
// ...
// You can tag a span
newSpan.tag("commissionValue", commissionValue);
// ...
// You can log an event on a span
newSpan.event("commissionCalculated");
// Once done remember to end the span. This will allow collecting
// the span to send it to e.g. Zipkin. The tags and events set on the
// newSpan will not be present on the parent
newSpan.end();

Micrometer Tracing Brave Setup

In this subsection, we set up Micrometer Tracing with Brave.

The following example shows how to create a Micrometer Tracing Tracer by using Brave components that would send completed spans to Zipkin:

// [Brave component] Example of using a SpanHandler. SpanHandler is a component
// that gets called when a span is finished. Here we have an example of setting it
// up with sending spans
// in a Zipkin format to the provided location via the UrlConnectionSender
// (through the <io.zipkin.reporter2:zipkin-sender-urlconnection> dependency)
// Another option could be to use a TestSpanHandler for testing purposes.
AsyncZipkinSpanHandler spanHandler = AsyncZipkinSpanHandler
    .create(URLConnectionSender.create("http://localhost:9411/api/v2/spans"));

// [Brave component] CurrentTraceContext is a Brave component that allows you to
// retrieve the current TraceContext.
ThreadLocalCurrentTraceContext braveCurrentTraceContext = ThreadLocalCurrentTraceContext.newBuilder()
    .addScopeDecorator(MDCScopeDecorator.get()) // Example of Brave's
                                                // automatic MDC setup
    .build();

// [Micrometer Tracing component] A Micrometer Tracing wrapper for Brave's
// CurrentTraceContext
CurrentTraceContext bridgeContext = new BraveCurrentTraceContext(this.braveCurrentTraceContext);

// [Brave component] Tracing is the root component that allows to configure the
// tracer, handlers, context propagation etc.
Tracing tracing = Tracing.newBuilder()
    .currentTraceContext(this.braveCurrentTraceContext)
    .supportsJoin(false)
    .traceId128Bit(true)
    // For Baggage to work you need to provide a list of fields to propagate
    .propagationFactory(BaggagePropagation.newFactoryBuilder(B3Propagation.FACTORY)
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span_in_scope 1")))
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span_in_scope 2")))
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span")))
        .build())
    .sampler(Sampler.ALWAYS_SAMPLE)
    .addSpanHandler(this.spanHandler)
    .build();


// [Brave component] Tracer is a component that handles the life-cycle of a span
brave.Tracer braveTracer = this.tracing.tracer();

// [Micrometer Tracing component] A Micrometer Tracing wrapper for Brave's Tracer
Tracer tracer = new BraveTracer(this.braveTracer, this.bridgeContext, new BraveBaggageManager());

Micrometer Tracing OpenTelemetry Setup

In this subsection, we set up Micrometer Tracing with OpenTelemetry (OTel).

The following example shows how to create a Micrometer Tracing Tracer by using OTel components that would send completed spans to Zipkin:

// [OTel component] Example of using a SpanExporter. SpanExporter is a component
// that gets called when a span is finished. Here we have an example of setting it
// up with sending spans
// in a Zipkin format to the provided location via the UrlConnectionSender
// (through the <io.opentelemetry:opentelemetry-exporter-zipkin> and
// <io.zipkin.reporter2:zipkin-sender-urlconnection> dependencies)
// Another option could be to use an ArrayListSpanProcessor for testing purposes
SpanExporter spanExporter = new ZipkinSpanExporterBuilder()
    .setSender(URLConnectionSender.create("http://localhost:9411/api/v2/spans"))
    .build();

// [OTel component] SdkTracerProvider is an SDK implementation for TracerProvider
SdkTracerProvider sdkTracerProvider = SdkTracerProvider.builder()
    .setSampler(alwaysOn())
    .addSpanProcessor(BatchSpanProcessor.builder(spanExporter).build())
    .build();

// [OTel component] The SDK implementation of OpenTelemetry
OpenTelemetrySdk openTelemetrySdk = OpenTelemetrySdk.builder()
    .setTracerProvider(sdkTracerProvider)
    .setPropagators(ContextPropagators.create(B3Propagator.injectingSingleHeader()))
    .build();

// [OTel component] Tracer is a component that handles the life-cycle of a span
io.opentelemetry.api.trace.Tracer otelTracer = openTelemetrySdk.getTracerProvider()
    .get("io.micrometer.micrometer-tracing");

// [Micrometer Tracing component] A Micrometer Tracing wrapper for OTel
OtelCurrentTraceContext otelCurrentTraceContext = new OtelCurrentTraceContext();

// [Micrometer Tracing component] A Micrometer Tracing listener for setting up MDC
Slf4JEventListener slf4JEventListener = new Slf4JEventListener();

// [Micrometer Tracing component] A Micrometer Tracing listener for setting
// Baggage in MDC. Customizable
// with correlation fields (currently we're setting empty list)
Slf4JBaggageEventListener slf4JBaggageEventListener = new Slf4JBaggageEventListener(Collections.emptyList());

// [Micrometer Tracing component] A Micrometer Tracing wrapper for OTel's Tracer.
// You can consider
// customizing the baggage manager with correlation and remote fields (currently
// we're setting empty lists)
OtelTracer tracer = new OtelTracer(otelTracer, otelCurrentTraceContext, event -> {
    slf4JEventListener.onEvent(event);
    slf4JBaggageEventListener.onEvent(event);
}, new OtelBaggageManager(otelCurrentTraceContext, Collections.emptyList(), Collections.emptyList()));

Micrometer Tracing Baggage API

Traces connect from application to application by using header propagation. Besides trace identifiers, other properties (called Baggage) can also be passed along with the request.

The following example shows how to use the Tracer API to create and extract baggage:

Span span = tracer.nextSpan().name("parent").start();

// Assuming that there's a span in scope...
try (Tracer.SpanInScope ws = tracer.withSpan(span)) {

    try (BaggageInScope baggageForSpanInScopeOne = tracer.createBaggageInScope("from_span_in_scope 1",
            "value 1")) {
        then(baggageForSpanInScopeOne.get()).as("[In scope] Baggage 1").isEqualTo("value 1");
        then(tracer.getBaggage("from_span_in_scope 1").get()).as("[In scope] Baggage 1").isEqualTo("value 1");
    }

    try (BaggageInScope baggageForSpanInScopeTwo = tracer.createBaggageInScope("from_span_in_scope 2",
            "value 2");) {
        then(baggageForSpanInScopeTwo.get()).as("[In scope] Baggage 2").isEqualTo("value 2");
        then(tracer.getBaggage("from_span_in_scope 2").get()).as("[In scope] Baggage 2").isEqualTo("value 2");
    }
}

// Assuming that you have a handle to the span
try (BaggageInScope baggageForExplicitSpan = tracer.createBaggageInScope(span.context(), "from_span",
        "value 3")) {
    then(baggageForExplicitSpan.get(span.context())).as("[Span passed explicitly] Baggage 3")
        .isEqualTo("value 3");
    then(tracer.getBaggage("from_span").get(span.context())).as("[Span passed explicitly] Baggage 3")
        .isEqualTo("value 3");
}

// Assuming that there's no span in scope
// When there's no span in scope, there will never be any baggage - even if you
// make it current
try (BaggageInScope baggageFour = tracer.createBaggageInScope("from_span_in_scope 1", "value 1");) {
    then(baggageFour.get()).as("[Out of span scope] Baggage 1").isNull();
    then(tracer.getBaggage("from_span_in_scope 1").get()).as("[Out of span scope] Baggage 1").isNull();
}
then(tracer.getBaggage("from_span_in_scope 1").get()).as("[Out of scope] Baggage 1").isNull();
then(tracer.getBaggage("from_span_in_scope 2").get()).as("[Out of scope] Baggage 2").isNull();
then(tracer.getBaggage("from_span").get()).as("[Out of scope] Baggage 3").isNull();

// Baggage is present only within the scope
then(tracer.getBaggage("from_span").get(span.context())).as("[Out of scope - with context] Baggage 3").isNull();

For Brave, remember to set up the PropagationFactory so that it contains the baggage fields that you will be using in your code. Check the following example for details:

Tracing tracing = Tracing.newBuilder()
    .currentTraceContext(this.braveCurrentTraceContext)
    .supportsJoin(false)
    .traceId128Bit(true)
    // For Baggage to work you need to provide a list of fields to propagate
    .propagationFactory(BaggagePropagation.newFactoryBuilder(B3Propagation.FACTORY)
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span_in_scope 1")))
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span_in_scope 2")))
        .add(BaggagePropagationConfig.SingleBaggageField.remote(BaggageField.create("from_span")))
        .build())
    .sampler(Sampler.ALWAYS_SAMPLE)
    .addSpanHandler(this.spanHandler)
    .build();

Baggage with Micrometer Observation API

If you’re using Micrometer Observation API, there’s no notion of baggage. If you set up a BaggageManager to have the baggage fields configured, we will assume that when the Observation gets put in scope, whatever low and high cardinality keys are set on the Observation will be put in scope as baggage (assuming that their names match with the configuration on the BaggageManager). Below you can find example of such setup with OpenTelemetry BaggageManager.

// There will be 3 baggage keys in total, 2 for remote fields and 1 as tag field
OtelBaggageManager otelBaggageManager = new OtelBaggageManager(otelCurrentTraceContext,
        Arrays.asList(KEY_1, OBSERVATION_BAGGAGE_KEY), Collections.singletonList(TAG_KEY));


// For automated baggage scope creation the tracing handler is required
observationRegistry.observationConfig().observationHandler(new DefaultTracingObservationHandler(tracer));

// An observation with low and high cardinality keys
// with key names equal to baggage key entries set on the baggage manager
Observation observation = Observation.start("foo", observationRegistry)
    .lowCardinalityKeyValue(KEY_1, TAG_VALUE)
    .highCardinalityKeyValue(OBSERVATION_BAGGAGE_KEY, OBSERVATION_BAGGAGE_VALUE);

// There is no baggage here
try (Scope scope = observation.openScope()) {
    // Baggage here will be automatically put in scope
    then(tracer.getBaggage(KEY_1).get()).isEqualTo(TAG_VALUE);
    then(tracer.getBaggage(OBSERVATION_BAGGAGE_KEY).get()).isEqualTo(OBSERVATION_BAGGAGE_VALUE);
}
// There is no baggage here

Aspect Oriented Programming

Micrometer Tracing contains @NewSpan, @ContinueSpan, and @SpanTag annotations that frameworks can use to create or customize spans for either specific types of methods such as those serving web request endpoints or, more generally, to all methods.

Micrometer’s Spring Boot configuration does not recognize these aspects on arbitrary methods.

An AspectJ aspect is included. You can use it in your application, either through compile/load time AspectJ weaving or through framework facilities that interpret AspectJ aspects and proxy targeted methods in some other way, such as Spring AOP. Here is a sample Spring AOP configuration:

@Configuration
public class SpanAspectConfiguration {

    @Bean
    NewSpanParser newSpanParser() {
        return new DefaultNewSpanParser();
    }

    // You can provide your own resolvers - here we go with a noop example.
    @Bean
    ValueResolver valueResolver() {
        return new NoOpValueResolver();
    }

    // Example of a SpEL resolver
    @Bean
    ValueExpressionResolver valueExpressionResolver() {
        return new SpelTagValueExpressionResolver();
    }

    @Bean
    MethodInvocationProcessor methodInvocationProcessor(NewSpanParser newSpanParser, Tracer tracer,
            BeanFactory beanFactory) {
        return new ImperativeMethodInvocationProcessor(newSpanParser, tracer, beanFactory::getBean,
                beanFactory::getBean);
    }

    @Bean
    SpanAspect spanAspect(MethodInvocationProcessor methodInvocationProcessor) {
        return new SpanAspect(methodInvocationProcessor);
    }

}

// Example of using SpEL to resolve expressions in @SpanTag
static class SpelTagValueExpressionResolver implements ValueExpressionResolver {

    private static final Log log = LogFactory.getLog(SpelTagValueExpressionResolver.class);

    @Override
    public String resolve(String expression, Object parameter) {
        try {
            SimpleEvaluationContext context = SimpleEvaluationContext.forReadOnlyDataBinding().build();
            ExpressionParser expressionParser = new SpelExpressionParser();
            Expression expressionToEvaluate = expressionParser.parseExpression(expression);
            return expressionToEvaluate.getValue(context, parameter, String.class);
        }
        catch (Exception ex) {
            log.error("Exception occurred while tying to evaluate the SpEL expression [" + expression + "]", ex);
        }
        return parameter.toString();
    }

}

Applying SpanAspect makes @NewSpan and @ContinueSpan usable on any arbitrary method in an AspectJ proxied instance, as the following example shows:

// In Sleuth @NewSpan and @ContinueSpan annotations would be taken into
// consideration. In Micrometer Tracing due to limitations of @Aspect
// we can't do that. The @SpanTag annotation will work well though.
protected interface TestBeanInterface {

    void testMethod2();

    void testMethod3();

    void testMethod10(@SpanTag("testTag10") String param);

    void testMethod10_v2(@SpanTag("testTag10") String param);

}

// Example of an implementation class
protected static class TestBean implements TestBeanInterface {

    @NewSpan
    @Override
    public void testMethod2() {
    }

    @NewSpan(name = "customNameOnTestMethod3")
    @Override
    public void testMethod3() {
    }

    @ContinueSpan(log = "customTest")
    @Override
    public void testMethod10(@SpanTag("customTestTag10") String param) {

    }

    @ContinueSpan(log = "customTest")
    @Override
    public void testMethod10_v2(String param) {

    }

}

// --------------------------
// ----- USAGE EXAMPLE ------
// --------------------------


// Creates a new span with
testBean().testMethod2();
then(createdSpanViaAspect()).isEqualTo("test-method2");

// Uses the name from the annotation
testBean().testMethod3();
then(createdSpanViaAspect()).isEqualTo("custom-name-on-test-method3");

// Continues the previous span
Span span = this.tracer.nextSpan().name("foo");
try (Tracer.SpanInScope ws = this.tracer.withSpan(span.start())) {

    // Adds tags and events to an existing span
    testBean().testMethod10("tagValue");
    SimpleSpan continuedSpan = modifiedSpanViaAspect();
    then(continuedSpan.getName()).isEqualTo("foo");
    then(continuedSpan.getTags()).containsEntry("customTestTag10", "tagValue");
    then(continuedSpan.getEvents()).extracting("value").contains("customTest.before", "customTest.after");
}
span.end();

// Continues the previous span
span = this.tracer.nextSpan().name("foo");
try (Tracer.SpanInScope ws = this.tracer.withSpan(span.start())) {

    // Adds tags and events to an existing span (reusing setup from the parent
    // interface)
    testBean().testMethod10_v2("tagValue");
    SimpleSpan continuedSpan = modifiedSpanViaAspect();
    then(continuedSpan.getName()).isEqualTo("foo");
    then(continuedSpan.getTags()).containsEntry("testTag10", "tagValue");
    then(continuedSpan.getEvents()).extracting("value").contains("customTest.before", "customTest.after");
}
span.end();