Skip to content
AITroveRead. Build. Understand.
Reading list
Make this comfortable

Scheduled Job Overlap, Cancellation, and Compensation

Last updated: 5 Oct 20267 min read
tutorial
IntermediateBy AITrove Editorial

A schedule specifies when work becomes eligible, not that a prior run has finished. If a new tick arrives while an old run is active, the system needs a skip, coalesce, queue, or parallel policy. Cancellation is another state transition: it can stop future steps, but it cannot automatically undo a file already delivered or a message already sent. Compensation is a separate business action that may be partial and must be visible. These policies belong to the job contract before a scheduler is enabled.

Working case

A nightly permit digest groups open case changes for each tenant. A large tenant takes longer than the interval and the next tick arrives. Parallel runs would email duplicate summaries and compete for the same database snapshot. The service coalesces ticks under a tenant and schedule window key, then records the exact source revision used. Reviewer 47 later cancels a manual report while rendering is in progress. The worker checks cancellation before publishing the artifact, but if a notification was already sent, the UI reports that fact and offers a corrective notice instead of pretending the send vanished.

Implementation boundary

javascript
function mayCommitScheduledRun(run) {
  return run.claimVersion === run.currentVersion && !run.cancelRequested;
}
console.log(mayCommitScheduledRun({ claimVersion: 4, currentVersion: 5, cancelRequested: false }));
// Output: false

Give each scheduled occurrence a stable window ID and unique tenant key. Use a database claim or lease with fencing to prevent stale workers from committing after a newer run owns the window. Define whether missed ticks are replayed after downtime and how far back the system is allowed to catch up. Cancellation writes a requested state; workers check it at safe checkpoints and commit a terminal canceled state only after they stop. Separate reversible steps, such as deleting an unpublished temporary artifact, from irreversible effects, such as an external email. A compensation action gets its own ID and audit record.

Cost and boundaries

Serializing one tenant reduces duplicate work but can increase delay for a large backlog. Coalescing may skip intermediate snapshots by design, so the digest must state its revision window. A claim lookup is O(log n) with an index across scheduled windows; scanning all tenants every second is wasteful. Cancellation checks add database reads, so place them at meaningful checkpoints instead of every rendered row. Compensation has operational cost and cannot guarantee that recipients forget a sent message. Measure overlap, skipped ticks, cancel latency, and already-escaped effects.

Failure trace

Pause one digest run before publish, fire the next schedule tick, and verify only one owner can commit for that window. Kill the worker after it sends an email but before state commit; replay must consult the effect ledger rather than send again. Request cancellation during rendering and confirm the unpublished artifact is removed. Request it after a notice was sent and show a partial-cancellation outcome. Simulate two days of scheduler outage; enforce the documented catch-up bound instead of launching an unlimited flood. Change tenant ownership during a long run and reject stale commit.

Verification

  • Only the current claim can publish a scheduled window.
  • Cancellation reports effects that already escaped.
  • Catch-up work respects a stated backlog bound.

Practice drill

Create a digest schedule for 83 tenants with a 47-minute window. Force one tenant to run longer than the interval and compare skip, coalesce, and queue policies. Choose one, record missed-tick behavior, and test worker fencing. Add manual cancel before and after the notification step. The status resource should distinguish stopped work, published effects, and any compensation still pending.

Decision note

A scheduler and a cancel button need explicit overlap and effect policies; neither implies work can be undone.

Common Mistakes

  • Running every missed tick at once after an outage.
  • Treating cancel requested as cancel completed.
  • Deleting evidence of an external send during compensation.

Related lessons

Background Workflow Reliability; Job Admission, Idempotency, and Status Resources; Worker Leases, Retries, and Duplicate Effects; Poison Job Quarantine and Replay Control; Export and Erasure Job State; Feature Release and Experiment Controls.

Connected practice

Build Project: permit report job recovery and review Web Development: jobs and abuse decisions quiz.

web-tech
web-development
Storage details