correlation tracing
why
a multi-hop operation leaves evidence in several services; the only thing that reconnects it is one stable id minted at the boundary that opened it. if each hop mints its own id, reconstruction is timestamp guessing across log files — thirty minutes to understand what took three seconds to execute. with one id, an incident is one query, not an archaeology dig.
rule
every operation gets exactly one correlation id, established at the outermost boundary, carried in execution context, written into every log line and every durable record the operation produces.
propagation map
| hop | carrier | rule |
|---|---|---|
| inbound request | response header / typed field | edge generates or adopts the caller's id, establishes context. |
| log lines | execution context | every line inside the operation carries it automatically. |
| outbound provider call | typed field in the call's log event | correlation id of the calling operation. |
| workflow/step handoff | durable handoff input | the id is snapshotted as a durable fact at handoff. |
| re-entry (callback/event) | re-established from the durable record | the re-entering boundary adopts the original id. |
| error response | typed diagnostic field | the person reporting a problem supplies the id that finds everything. |
extends async local execution context: context is established once at boundaries; the correlation id is its first field. detached callback context records the failure mode when re-entry forgets to re-adopt it.
do not
- generate a fresh correlation id per service call — one operation, one id.
- log through a path that bypasses context (raw console, ad-hoc logger instances without context).
- hand off to a workflow without snapshotting the correlation id in the durable input.
- return an error to a person without the correlation id in the diagnostic.
verification
take one recent incident-ish request (or run one probe). query logs by its correlation id: every line — edge, service, provider call, step retry — must appear under that single id, including across a workflow re-entry. confirm the error response the person sees contains the same id.
good and bad
bad
await provider.charge(input); // logs under a fresh request idawait step.run("record", () => recordCharge(input)); // a new id again
each hop mints its own id; the person reporting the failure can supply only one of them, and it finds exactly one hop of a multi-hop path — the rest stays unrecoverable, and cross-service questions like "did the provider retry hit our records" are answered by conjecture.
good
// edge establishes context once; the handoff snapshots the id as durable factawait step.run("record", (ctx) => recordCharge({ ...input, correlationId: ctx.correlationId }));
one id from the boundary, carried by context and snapshotted into the durable handoff — reporter, engineer, and durable record share the same join key, so the whole operation is one lookup, at flat cost however many hops the architecture grows.