Connection Pool Budgets and Queue Admission

Working case

A permit service runs six API replicas, each configured for 29 connections. Two report workers each open 17 more, and an operator reserves 12 slots for maintenance. The resulting possible demand is 220 connections. The database allows 190 ordinary application connections. During a field inspection upload, a remote image processor pauses while holding a transaction and connection. Requests queue behind full pools, application memory rises, and a rollout doubles replica count. The fix starts by counting all owners, setting per-process budgets under the database ceiling, releasing connections before network calls, and placing a short acquisition deadline on the queue.

Implementation boundary

function totalDatabaseSlots(apiReplicas, apiPool, workerReplicas, workerPool, reserved) {
  return apiReplicas * apiPool + workerReplicas * workerPool + reserved;
}
console.log(totalDatabaseSlots(6, 29, 2, 17, 12) <= 190);
// Output: false

Write a connection inventory as part of the deployment plan: maximum replicas times API pool size, plus workers, scheduled jobs, migrations, and explicit spare capacity. Ensure autoscaling cannot exceed that envelope without changing the budget. Keep transactions short and acquire the connection close to the query, rather than at the start of a long HTTP request. Put a finite bound on pool waiters and a deadline on acquisition; return an intentional overload response or shed optional work once admission is full. If an intermediary pooler is used, verify whether it pools sessions or transactions, because session state, prepared statements, and temporary tables may behave differently. Keep health checks from consuming the last useful slot. Test an ordinary rollout and a failed region that moves traffic to fewer healthy instances.

Cost and boundaries

A smaller pool can reduce database thrash and memory use, but it may cap useful concurrency when queries are short and the database has spare capacity. A larger pool can raise query concurrency while also increasing contention, context switching, and the blast radius of a stalled query. Queueing adds latency before execution; an unbounded queue only postpones the error while using application memory. Track pool active and idle counts, acquisition p95, rejected waiters, database connections, transaction duration, and lock wait time. Multiply by replica count when interpreting every metric. Measure at a known arrival rate instead of choosing a pool size by CPU count alone.

Failure trace

Increase API replicas from six to nine without changing their pool size. The aggregate connection ceiling is crossed even though each replica's health check passes. Next, hold one connection across a 47-second image-service call; a burst fills the remaining pools and makes unrelated case reads wait. Kill one region and route its traffic into the survivors while the database limit stays fixed. Verify that admission rejects excess work promptly, maintenance can still connect, and queued requests do not wait beyond their caller deadlines. A pool timeout must be distinguishable from a slow SQL query in telemetry.

Verification

Fleet-wide maximum stays within the tested database connection ceiling.
Pool acquisition has a finite deadline and bounded waiting.
An external service pause does not hold a database connection.

Practice drill

Set a database application limit of 190, reserve 12 slots, six API replicas at 29 each, and two workers at 17 each. Compute the initial overage. Choose new limits that leave room for one additional API replica during rollout and for emergency access. Run a load test with one intentionally slow query and one slow external call. Capture acquisition time, query time, lock wait, queue length, and rejection count. Then drain a replica and verify that released requests are retried only when the operation permits it.

Decision note

Pool size is a fleet-level capacity decision; waiting needs a budget and an exit path.

Common Mistakes

Sizing one process pool without multiplying by replicas.
Treating a pool queue as extra database capacity.
Keeping a connection during slow network or user work.

Apply and check

Build Project: permit data change under live traffic and review Web Development: database online operations quiz.

Further connections

Cache Memory, Eviction, and Degraded Read Paths.