Logs, metrics, and traces answer different questions. A structured event can preserve one operation outcome with a bounded event name, release, route template, duration, and error category. A metric aggregates counts or distributions across a limited set of labels. A trace links timed operations. None should ingest arbitrary user notes by default. Define an allowlist of fields for each signal at the point of creation; removing a secret after it reached a collector may be too late. Retention and access controls belong to the signal contract. Cardinality is the number of distinct label combinations, and it can rise multiplicatively when independent labels are combined.
Make this comfortable
Telemetry Shapes, Redaction, and Cardinality
Working case
A case-47 save fails only for release 29. A developer adds case ID, reviewer email, URL, and raw validation body to every metric label. One release produces thousands of series and exposes private note text in a monitoring system. A better design uses the route template, release identifier, outcome class, and bounded status code on the metric. The structured event retains a short internal operation ID under a shorter retention policy. A sampled trace shows which API or worker stage slowed down. Private payloads stay in the application record with normal access control.
Implementation boundary
function diagnosticFields(event) {
const allowed = ["operationId", "route", "release", "outcome"];
return Object.fromEntries(allowed.filter(key => event[key] !== undefined).map(key => [key, event[key]]));
}
console.log(JSON.stringify(diagnosticFields({ operationId: "op-47", route: "/cases/:id", release: "r29", outcome: "failed", note: "private" })));
// Output: {"operationId":"op-47","route":"/cases/:id","release":"r29","outcome":"failed"}The allowlist is an illustration. Production collection should also validate value length and shape, scrub exception messages that may contain payloads, and enforce a bounded enumeration for metric labels. Keep raw operation IDs out of metric dimensions, though short-lived logs may need them for correlation. Decide who can read each telemetry store and how long data remains there. Test the final exported payload after SDK and proxy enrichment, because automatic instrumentation can add headers or URLs that the application did not explicitly log. Redaction must happen before export, not only in a dashboard view.
Cost and boundaries
Writing one event is O(1) application work, but exporting and indexing n events consumes O(n) downstream storage. If a metric has label sets with sizes a, b, and c, the possible series count can approach a × b × c; even one unbounded label can dominate cost. Sampling traces saves storage but weakens single-request coverage. Very short retention lowers exposure and cost, yet can erase evidence before a delayed incident is discovered. Set retention by investigation need and data sensitivity, and test queries under realistic incident volume. Measure dropped events as its own bounded signal.
Failure trace
An exception handler records error.stack plus serialized request.body. A failed upload now copies private case descriptions into a tracing service with broad employee access. An application-level filter removes body, but a reverse proxy still adds the full URL containing a query token. Review the complete exported event, configure auto-instrumentation, and replace raw URLs with route templates. Another failure gives every retry a unique error-name label; series counts grow until queries time out during the incident. Use stable error categories and keep detailed identifiers in access-controlled events.
Verification
- Private payload markers do not reach any exported telemetry signal.
- Metric series remain bounded as case count grows.
- Release and route can still be isolated during a diagnosis.
Practice drill
Submit a case note containing a marker that looks like a secret, then inspect logs, spans, metric labels, and collector exports for that marker. Generate 47,000 distinct case IDs and verify metric-series count remains close to the bounded route/outcome combinations. Cause a validation exception that includes input text and check that the exported error category remains useful without copying the text. Expire test events according to retention policy, then simulate a delayed incident and confirm the surviving aggregate evidence is sufficient to identify release and route.
Decision note
Allowlist and bound telemetry at creation, and keep detailed correlation out of metric dimensions.
Common Mistakes
- Logging an entire request object for convenience.
- Treating dashboard redaction as source redaction.
- Using unique operation IDs as metric labels.
Connected lessons
Production Signals and Incident Decisions; Trace Context Across Requests and Jobs; User Journey SLOs and Burn Alerts; Incident Containment and Evidence Timeline; Telemetry Minimization and Retention; Browser Security and Data Stewardship; Trace Context Across Requests and Jobs.
Apply and check
Build Project: case-save incident evidence and review Web Development: operations and sync decisions quiz.
Further connections
web-tech
web-development