Skip to content
AITroveRead. Build. Understand.
Reading list
Make this comfortable

Payment Webhook Reconciliation and Idempotent Fulfillment

Last updated: 4 Oct 20266 min read
tutorial
IntermediateBy AITrove Editorial

Payment providers send asynchronous status events, and delivery is usually retryable. A webhook is evidence only after its signature, account scope, event identity, and payload are validated under the provider’s contract. Even a verified event may be duplicated or arrive after a newer state. Fulfillment must therefore be an idempotent, server-owned transition tied to a confirmed payable amount and order. The browser return page can observe this transition but cannot trigger it by assertion. Where an event is ambiguous, fetch the provider’s current status before finalizing the order.

Working case

Order 47 is awaiting a 17,200-unit capture. The provider sends event E-62 twice, then retries E-47 from an earlier authorization state. The handler verifies each raw event, records unique event IDs, checks the provider payment ID and amount against order 47, and advances from pending to captured once. A unique fulfillment key allows one warehouse release. The duplicate E-62 is acknowledged without a second shipment; older E-47 cannot move the order backward. If the webhook never arrives, a scheduled reconciliation query finds the provider’s captured state and applies the same transition.

Implementation boundary

javascript
function fulfillmentKey(orderId, paymentId) {
  return `fulfill:${orderId}:${paymentId}`;
}
console.log(fulfillmentKey(47, "pay-62"));
// Output: fulfill:47:pay-62

Preserve the raw request bytes required by the provider’s signature scheme and verify before parsing or trusting fields. Check event ID, provider account, payment object, currency, amount, and relation to the local order. Store processed event IDs under a uniqueness constraint and perform state transition plus fulfillment-outbox insertion in one transaction. A worker consumes the outbox with its own idempotency key. Acknowledge duplicates deliberately, retry transient internal failures, and quarantine mismatches for review. Use a reconciliation job for missing or delayed events; do not assume webhook order reflects payment-state order.

Cost and boundaries

Signature verification is bounded cryptographic work. Indexed event and payment lookups are near O(1), while every event adds a small deduplication record and audit trail. An outbox adds a write and worker hop but prevents a crash between paid-state update and shipment request from losing work. Reconciliation costs provider reads and must respect rate limits; sweep unresolved attempts with backoff rather than querying every historical payment. Track capture-to-fulfillment delay, duplicate events, mismatched amounts, unresolved attempts, and repeated outbox delivery. Acknowledging too early can lose a transition; acknowledging too late can flood retries.

Failure trace

A handler updates the order to paid, then crashes before shipping. The provider retries, but the handler sees paid and exits, leaving an unfulfilled order. Write the state change and outbox item atomically, then retry the worker. Another handler fulfills immediately from a browser success callback and ships twice when the webhook arrives. Keep one authoritative transition and unique fulfillment key. Test duplicate events, reversed event order, bad signature, wrong account, wrong amount, missing webhook, provider query disagreement, database failure before commit, and worker crash after an external shipment call.

Verification

  • Duplicate and out-of-order events cannot duplicate fulfillment.
  • Amount, currency, account, and signature are checked before mutation.
  • Missing events can be reconciled through the same transition.

Practice drill

Send a signed capture event for order 47, then replay it five times and reorder it with an older authorization event. Confirm one paid transition and one fulfillment intent. Send a validly signed event for a different amount and verify quarantine. Stop the webhook worker, let provider status become captured, and run reconciliation. Crash the fulfillment worker after its external request, retry it, and confirm the warehouse receives one effective instruction by key.

Decision note

Verify provider evidence and use one idempotent state transition for both webhook and reconciliation paths.

Common Mistakes

  • Parsing before a raw-body signature check required by the provider.
  • Assuming delivery order is payment-state order.
  • Updating paid status separately from fulfillment intent.

Connected lessons

Checkout and Payment State; Server-Priced Order Snapshots and Money Units; Hosted Checkout Sessions and Browser Returns; Refunds, Reversals, and Payment Ledger State; Signed Webhook Delivery and Replay Control; Transactions and Concurrent Writes; Idempotent Write Requests and Lost Responses.

Apply and check

Build Project: checkout reconciliation and review Web Development: passkey and checkout decisions quiz.

web-tech
web-development
Storage details