Distributed traces: preserve context without leaking data

Operational decision

A parcel API calls a pricing service and a document renderer. Propagate trace context through the approved library rather than manually copying an unvalidated header. Record stable operation names and service identity; put a bounded deployment version attribute on spans so an incident can compare old and new releases. Do not attach full addresses or customer names. The shell request below supplies a syntactically valid traceparent header to a local diagnostic endpoint; it tests propagation only when both services export spans to an observable backend. Inspect whether all three spans share the trace ID and whether the edge sanitizes incoming context according to the trust policy. If sampling drops the request, a missing trace is not proof no work occurred; logs and metrics must still describe the user failure.

curl --fail --show-error \
  -H 'traceparent: 00-c4e2b6114e3047bb91ef681ca5f8a370-b84c2271f8a3d965-01' \
  127.0.0.1:8147/diagnostics/parcel-price

Cost and verification

Traces consume network and storage proportional to request volume, span count, and sampling policy. Head sampling is cheap but may miss rare failures; retaining every trace can be expensive. Keep span attributes low in cardinality when aggregating and set retention appropriate to incident review. Propagating internal trace identifiers to outside endpoints can reveal architecture details, so decide where context is stripped. A trace is a debugging record, not a security token or proof of business success.

Common Mistakes

• Do not put secrets or personal data in baggage.
• Do not trust incoming trace headers as authorization.
• Do not conclude that an unsampled request never happened.

Connected lessons

• DevOps: delivery, infrastructure, and reliable operations
• Observability: join metrics, logs, and traces
• Alert design: page on impact and include a first action
• Incident response: contain impact, then learn

Advanced follow-up

• Trace sampling: retain useful failures without flooding storage

Advanced follow-up

• Collector export queues: measure the failure budget before telemetry is dropped