Expensive Request Work Budgets and Load Shedding

Working case

The permit search endpoint accepts a page size and many filters. A client asks for 47,000 rows and a count for every status group. The endpoint spawns a broad database scan and serializes a huge response while regular reviewers wait. Another client submits a small page size but repeats nested filters that make the query expensive. A simple requests-per-minute counter misses both. The service caps page size, filter count, execution time, concurrent expensive searches, and per-tenant budget. Larger reports move to an admitted background job with a status resource.

Implementation boundary

function searchCost(query) {
  return 1 + query.filterCount * 2 + (query.includeGroups ? 8 : 0);
}
console.log(searchCost({ filterCount: 5, includeGroups: true }));
// Output: 19

Validate the size and shape of every input before allocating work. Estimate query cost from bounded dimensions, then enforce a per-request ceiling and a separate tenant concurrency quota. Set database statement timeouts and response byte limits; cancel downstream work when the client disconnects where safe. Keep a small reserved capacity for ordinary case reads so bulk exports cannot starve them. Charge expensive third-party sends to a provider budget and return a clear retry or delayed-job contract when full. Avoid an unlimited queue that only moves overload into a later collapse. Record budget denials by category without private query strings.

Cost and boundaries

A cost estimator should be O(number of provided filter terms), with a strict bound on that count. Exact query planning can be expensive, so use conservative categories for admission and measure actual CPU, rows read, bytes, and provider charges afterward. Concurrency limits trade peak throughput for predictable latency. A queue absorbs short bursts only when workers and queue age are bounded; otherwise it becomes hidden debt. Track p95 task completion, admitted versus denied cost units, queue age, and error rates for normal low-cost requests.

Failure trace

Send a request with a huge page size and require rejection before database work. Submit many small but expensive filter combinations and confirm the tenant cost budget engages. Kill a client connection during a long search and check that unused downstream work is canceled. Saturate export workers and verify ordinary case reads still complete. Make a provider billing limit trip and stop optional sends while preserving core task state. Try to bypass limits by splitting one large operation into many parallel calls; the aggregate budget should still bind them.

Verification

• Inputs are bounded before allocation.
• Bulk work cannot starve ordinary case reads.
• Aggregate tenant work remains limited across parallel calls.

Practice drill

Define cost units for case search: base 1, each filter 2, group count 8, and export 31. Admit only requests under a per-call threshold and a per-tenant concurrent budget. Run 83 parallel requests from one tenant and measure ordinary read latency. Move the largest report to a job, then bound queue age and give the user a status path. Review false denials for legitimate analysts.

Decision note

Budget the work induced by a request, not just the request count.

Common Mistakes

• Using only requests per minute for costly queries.
• Letting a queue grow without age or size limits.
• Continuing expensive downstream work after cancellation.

Related lessons

Abuse-Resistant Public Endpoints; Account Recovery Enumeration and Throttle Policy; Human Challenges, Accessible Alternatives, and Signal Retention; Abuse Decision Telemetry and False-Positive Rollback; Search Query Normalization and Ranking; Quality and Capacity Engineering.

Connected practice

Build Project: public permit endpoint abuse controls and review Web Development: jobs and abuse decisions quiz.