Skip to content

Record an error on a span

Call spanerr.RecordFailure(ctx, err) with the request's context. The span in ctx gets an exception event carrying the error's go/errors kind, its message, the stack from where the error was created, and its hints and details, and the span's status becomes Error:

mux := http.NewServeMux()

mux.HandleFunc("POST /orders/{order}/charge", func(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()

    if err := charge(ctx, r.PathValue("order")); err != nil {
        spanerr.RecordFailure(ctx, err)
        http.Error(w, "payment failed", http.StatusPaymentRequired)

        return
    }

    w.WriteHeader(http.StatusNoContent)
})

spanerr imports the OpenTelemetry APIs and go/errors, and not the SDK or any exporter, so a library can call it without putting an exporter into every binary that imports the library.

The span has to be in ctx. Instrumentation such as otelhttp puts it there for a request. If no span is in ctx, or the span is not being sampled, nothing is recorded on a span, and the call costs a few nanoseconds.

Record without failing the span

An error you recover from, such as a retry that later succeeds, can be recorded without marking the span failed:

spanerr.Record(ctx, err)

To fail the span without recording an event, for example because the error was already recorded further down, use spanerr.Fail(ctx, err). RecordFailure is Record followed by Fail.

Give the error an identity first

exception.type comes from errors.KindOf, so it names the failure only when the error has a kind. An error from errors.New records errors.basic, and one from errors.Newf, a wrap of a foreign error, or fmt.Errorf records no type at all. Declare a sentinel for each failure you want to tell apart in a trace backend:

var errCardDeclined = errors.NewSentinel("payment.card_declined", "the card was declined")

return errors.Wrapf(errCardDeclined, "charging order %s", orderID)

That records exception.type as payment.card_declined, whatever wraps it on the way up. Hints, details and attributes added with errors.WithHint, errors.WithDetail and errors.WithAttrs go on the event too. Exception attributes lists every key.

Send exceptions as log-based events

OpenTelemetry is moving exceptions from span events to log records. spanerr follows semconv's switch for that move, OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN:

OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN=logs/dup   # span event and log record
OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN=logs       # log record only

The log record goes through the global logger provider, so register one at start-up:

lp, err := logs.NewProvider(ctx, res, settings)
if err != nil {
    return err
}

otel.SetLoggerProvider(lp)

otel is go.opentelemetry.io/otel. The record carries the span's trace and span IDs. It is written even when the span is not sampled, because a log pipeline keeps records that a trace sampler drops.

The variable is read once, the first time spanerr records an error. Changing it after that has no effect until the process restarts.

Name the exception after the operation

semconv recommends that a log-based exception event be named after the operation that failed, with an .exception suffix. The default is plain exception. Middleware that knows the operation can name every exception recorded beneath it:

namePaymentExceptions := func(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ctx := spanerr.ContextWithEventName(r.Context(), "payments.charge.exception")
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

_ = namePaymentExceptions

A single call can override the name with an option:

spanerr.RecordFailure(ctx, err, spanerr.WithEventName("db.client.operation.exception"))

The name applies to the log record only. The span event is always called exception, as semconv requires.

Check what was recorded

In a test, build a tracer provider with the SDK's tracetest.SpanRecorder, run the code under a span from it, and read the event back from recorder.Ended(). The event's name is exception, and its attributes are the ones listed in Exception attributes.

The design and every decision behind it are in spec 0002, render an error onto a span.