A correlation ID across two services: join logs correctly, and do not confuse it with tracing Intermediate

The problem

A create-order request goes through the API and then calls a pricing service. With thousands of concurrent requests, one request’s log lines are mixed in with everyone else’s. Close timestamps do not prove they belong to the same request, and the orderId may not exist yet when the request starts.

A correlation ID is an identifier attached to one processing flow so you can look up its related logs.

The API creates the ID, puts it in logs, and sends it in a header when calling downstream; downstream logs the ID it received. This is the actual output of the demo in the repo (one request, two services, one ID):

{"event":"api.start","requestId":"a0638d3f-f7ea-4ef0-87a6-15734e1f3ad0"}
{"event":"downstream.received","requestId":"a0638d3f-f7ea-4ef0-87a6-15734e1f3ad0"}
{"event":"api.after-downstream","requestId":"a0638d3f-f7ea-4ef0-87a6-15734e1f3ad0"}

One requestId:"a0638d3f..." query in the log aggregator returns all three lines, even though the logs are in two services.

Which kind of ID to choose

	x-request-id	traceparent	baggage
Carries	One ID for finding logs	trace-id + parent-id + flags	Business key-value pairs
Standard	A convention, not a standard	W3C Trace Context (Recommendation 2021)	W3C Baggage (Candidate Recommendation)
Who creates it	Ingress/gateway	Tracing SDK (OpenTelemetry)	The application
Shows the span tree, latency	No	Yes, with spans	No
Use when	No tracing yet	Tracing is in place	You need to pass context such as tenant or tier

The rule: if you already run OpenTelemetry, use traceparent and put trace_id in logs; do not hand-roll another header. x-request-id only still has a reason to exist at ingress, as an ID returned to the client for support lookups.

The traceparent format (W3C): four fields separated by hyphens.

traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01
             │  └──── trace-id (32 hex) ───────┘ └─ parent-id ──┘ └ flags
             version (00)                         (16 hex)          (01 = sampled)

• An all-zero trace-id or parent-id is invalid.
• tracestate comes along to carry each tracing vendor’s own data.
• baggage can contain user-identifying data and travels to every downstream system; do not put sensitive data in it, and filter it at trust boundaries.

Core concepts

AsyncLocalStorage keeps a store bound to an asynchronous chain within one process; the log() function reads requestId without it being passed as a parameter.

That store does not travel over HTTP, a broker or to another service by itself: the code making the HTTP call must set the header, and the receiver reads it and attaches it to its own context. For the mechanism in detail, see AsyncLocalStorage.

One request ID is placed in the internal context and passed through an HTTP header

Enlarge diagram

Client → APIrequest

◀ Prev▶ PlayNext ▶↺ RestartStep 1/4

A request reaches the API. The API creates or validates a correlation ID at the trust boundary.

A header from the client is untrusted input: it may be very long, contain newlines or PII, or duplicate another request’s ID. The edge creates a new ID, or validates it with a regex and a length limit; the client’s ID, if needed, is stored in a separate field.

Propagation through Kafka

A message queue has no HTTP “request”, so the ID must travel in a record header; the consumer reads it back and restores the context.

// producer
await producer.send({ topic: 'orders', messages: [{
  key: orderId, value: payload,
  headers: { traceparent: currentTraceparent(), 'x-request-id': requestId },
}] });

// consumer
const requestId = message.headers?.['x-request-id']?.toString();
context.run({ requestId }, () => handle(message));   // logs in the handler carry the ID

With OpenTelemetry, the HTTP and Kafka client instrumentations usually inject/extract traceparent automatically; verify by capturing outbound headers in an integration test.

When to use it

Use	Not enough
Joining one request’s logs across services	You need per-hop latency: use a trace
Support lookups by the ID returned in the response	You need to follow a workflow spanning many requests: add a business ID or baggage
The system has no tracing yet	Using it as an idempotency key or for access control: wrong purpose

Real-world example

Support investigates an order failure

The gateway creates the ID, returns it in the x-request-id response header and puts it in the logs of API, Pricing and Inventory. The customer reports the error along with the ID; support filters logs by the ID, then opens the trace to see which hop was slow or failed.

The team does not use the request ID as a transaction key, because the same order may be retried with a new request. Nor do they put the ID in a metric label (see metric cardinality).

Run the demo from the repo root (two loopback servers, no Docker needed):

npx --prefix examples ts-node examples/src/correlation-id/correlation-demo.ts

Try removing the header in fetch: the downstream log loses the ID. Try removing context.run: the API log loses the ID. The two mechanisms are independent. The demo only proves one HTTP hop; there is no retry, broker or OpenTelemetry yet.

🎮 Try it

• Log, metric or trace?: gather one request’s logs across multiple services.

Production checklist

1. Ingress creates a new ID, or validates the one it receives (regex, at most about 64 characters, no newlines).
2. Echo the ID in a response header for clients and support.
3. Use one consistent log field name (request_id, trace_id) in every service.
4. Wrap the HTTP client and Kafka producer to forward the header; consumers restore the context.
5. Do not use a global variable for the ID; use AsyncLocalStorage.
6. With OpenTelemetry in place, prefer traceparent and log trace_id and span_id.
7. Test two parallel requests, and test that a downstream timeout still keeps the ID in the error log.

Trade-offs

Useful for

• Joining the logs of one processing run across hops without guessing by timestamp.
• AsyncLocalStorage removes the need to pass the ID through every function signature.
• Explicit headers are easy to test at the network boundary.

Limits and costs

• One service forgetting to forward breaks the chain.
• A request ID has no spans, latency or parent-child relationships.
• Logs still need retention, masking and access control.

Common mistakes

Common pitfall

Assuming AsyncLocalStorage propagates across the network by itself. The HTTP client must inject the header; the receiver must extract it and build its own context.

Common pitfall

Trusting every ID the client sends. A public header is easy to spoof to pollute logs. Create or normalize the ID at the trust boundary.

Summary

Key takeaways

• A correlation ID helps find the logs of one flow; it is not an access right or an idempotency key.
• traceparent is 00-<trace-id 32 hex>-<parent-id 16 hex>-<flags>; with OpenTelemetry in place, use it instead of a custom header.
• Over HTTP and Kafka the ID must be forwarded explicitly (headers); baggage is for business context.
• Validate the ID at ingress and echo it in the response.

Self-check: (1) If you remove x-request-id from fetch, which log loses the ID? (2) How many hex characters long is the parent-id field of traceparent? (3) How does a message going through Kafka carry the ID?

References

• Async context tracking — Node.js
• Context propagation — OpenTelemetry
• Trace Context — W3C
• Baggage — W3C