RED, USE and cardinality¶
Two methods for two kinds of thing¶
RED (rate, errors, duration) describes work: requests, queries, operations. USE
(utilisation, saturation, errors) describes resources: a connection pool, a queue, a
cache. instrument provides ready-made sets for both, because a raw metric.Meter leaves
every service to invent its own names, units and attributes.
Pull and push¶
go/transport-metrics serves these
same helpers to Prometheus, which scrapes them. instrument records them on an
OpenTelemetry meter, and the SDK pushes them over OTLP. A service usually wants one or the
other. The helpers share their method names so that moving between them means changing the
constructors and little else.
The two differ where each ecosystem has an opinion, and instrument follows OpenTelemetry
on those:
- Metric names are dotted, and use semantic conventions where they exist.
- Errors are an attribute. OpenTelemetry's conventions put
error.typeon the duration histogram and have no error counter, so the histogram's count gives both rate and errors. - Methods that record an event take a context. A measurement carries an exemplar only if a context reaches it, so a counter without one could never link to a trace.
The metric name map lists every difference.
What is not here¶
OpenTelemetry already covers the rest of what transport-metrics builds, so instrument
leaves it alone:
- HTTP and gRPC request metrics come from
otelhttpandotelgrpc. - Exemplars come from the SDK, with no configuration, when a sampled span is active.
- The version comes from the Resource, so there is no
build_infogauge. - Runtime metrics are one line, from
go.opentelemetry.io/contrib/instrumentation/runtime:
It is not wrapped here because that package links the SDK, and instrument must stay
importable from a library without doing so.
Keeping attribute values bounded¶
Each distinct attribute value is a new series, and series are what a metrics backend charges for in memory and money. Names passed to a helper are the caller's responsibility: use a small fixed set, never a per-call value such as an ID or a raw SQL string.
The SDK provides the backstop. Past a limit per instrument (2,000 by default), it folds
further values into one series marked otel.metric.overflow=true. The limit protects the
process, and it does not protect the data, because a noisy attribute can crowd out the rest
of its instrument. The application that builds the MeterProvider sets it, with
WithCardinalityLimit.
Errors are the one place the helper decides for you. error.type is a go/errors kind, which
is a constant in code, or _OTHER. An error's message never becomes an attribute, because
messages are unbounded by construction.
Metric names are a contract¶
Renaming a metric breaks dashboards and alerts even when no Go code changes, so emitted names change only deliberately, with a changelog note. Semantic-convention names come from a package whose import path carries the convention's version, so a name can only move when someone edits that import.