Project: assignment panel release experiment

Last updated: 5 Oct 20268 min read

project

Build a staged release for the case-review assignment panel. The service evaluates a versioned flag for stable reviewer IDs, exposes the new panel to staff and then a bounded cohort, and returns one resolved variant to the browser. Every assignment mutation still checks current case and tenant authorization. Reviewer 29 may see the new panel while reviewer 47 sees the old one, yet both use the same revisioned write contract. Define a safe evaluation default when configuration is unavailable. Write a stop rule for save failures and latency, then rehearse disabling the panel after new-form data exists. The old form must read those cases. For a separate experiment, assign a stable unit, record exposure only when either panel actually renders, and join an approved completion metric without private case fields. Check group balance, crossover, missing events, and errors before interpreting any result. When the release decision is settled, remove the temporary branch and ruleset while retaining the named emergency control.

Build contract

Evaluate the same reviewer across refresh, tabs, and devices, with a tested configuration-outage default.
Call the mutation endpoint directly as a reviewer without case rights; the flag must not grant access.
Create cases through the new form, turn the release off, and read them through the old form.
Compare control and treatment exposure at the same render point and detect missing or crossed assignments.

Implementation checkpoint

function releaseDecision(failureCount, attemptCount, maximumRate) {
  return attemptCount && failureCount / attemptCount > maximumRate ? "stop" : "continue";
}
console.log(releaseDecision(7, 47, 0.1));
// Output: stop

Cost and boundaries

Local evaluation is near O(1), while a remote flag lookup or refresh adds latency and a failure path. Supporting two forms doubles part of the test surface and requires state compatibility during rollout. Exposure logging adds records proportional to eligible rendered sessions, so minimize context and deduplicate repeated renders. The experiment needs enough data and a fixed observation rule before interpreting results; a tiny cohort may establish safety without supporting a business conclusion. Monitor variant-specific save errors, latency, config age, exposure joins, and the time required to stop the new path.

Failure drill

Set the flag to new in a browser header and call updateCase as reviewer 47 without permission; the server must reject it. Disable the new panel after reviewer 29 writes a new field and confirm the old form still opens the case. Suspend a tab during a kill action and try to submit its stale form; server evaluation must enforce the off state. Crash only the treatment before exposure logging and detect group imbalance rather than reading a biased metric. Finally remove the release flag and test offline clients, old stored values, and queued jobs.

Acceptance checks

Release exposure never replaces current object authorization.
The off path handles data created while the release was on.
Experiment groups have stable units and equivalent exposure points.
Temporary flag code and rules have a reviewed retirement path.

Common Mistakes

Calling a percentage rollout an experiment without data-quality checks.
Assuming a flag turn-off reverses completed writes.
Leaving an old path and remote rule after the decision is final.

web-tech

web-development