Backpressure is the mechanism that prevents a fast producer from overwhelming a slower consumer. A stream pipeline can propagate demand when each stage waits for the next stage to accept work. Application code breaks that property if it reads all chunks immediately and starts unlimited asynchronous processing. Await a bounded sink, cap queued bytes or records, and decide what to do when the downstream UI is slow. Dropping data is acceptable only for a protocol that explicitly permits it; a financial or audit feed needs ordered complete records or a visible gap. Use a single reader owner and cancel it when the page leaves.
Make this comfortable
Stream Backpressure and Bounded Work
Working case
A case activity feed sends 47,000 events. Parsing is quick, but each event triggers a database lookup in an enrichment service and a DOM update. A loop that calls enrich(event) without awaiting it accumulates thousands of promises and keeps every payload in memory. Limit concurrent enrichments, preserve sequence when results complete out of order, and commit visible rows in batches. If the user switches to case 62, abort the fetch and stop processing case 47's queued work. The server should also stop production when the connection closes if it can observe cancellation.
Implementation boundary
function withinQueueBudget(queuedBytes, nextBytes, maximumBytes) {
return queuedBytes + nextBytes <= maximumBytes;
}
console.log(withinQueueBudget(6200, 4700, 10000));
// Output: falseThe predicate expresses a byte limit; actual backpressure requires control flow that pauses reads or waits for a sink. For a ReadableStream pipeline, use a TransformStream or an awaited read/process loop so downstream readiness can influence upstream pulls. A manual queue needs both a maximum item count and a maximum byte count because one item may be huge. An event source that cannot pause must have a documented overflow policy, such as closing and resuming from a sequence token. Keep errors, cancellation, and completion distinct, and do not report completion while background tasks still run.
Cost and boundaries
Processing n records remains O(n) work, while a bounded queue of q records uses O(q) memory rather than O(n). A smaller q protects memory but may reduce throughput if processing stalls; a larger q absorbs bursts but increases latency and retention. Concurrent enrichment adds capacity only until the backend or browser saturates. Measure peak queue bytes, time to first usable batch, total completion, and responsiveness under a slow consumer. A single oversized frame should be rejected or separately handled, not admitted merely because queue item count is low.
Failure trace
The reader awaits response.body, then schedules each record with an async callback inside forEach. The outer function returns success immediately while thousands of callbacks remain pending. Memory rises and errors become detached from the feed state. Replace it with an awaited loop or bounded worker pool, and attach every task to the current route generation. Another failure is a queue limit measured only in records: one 40 MB record consumes most available memory while the count remains one. Enforce byte and frame limits together.
Verification
- Peak queue memory stays within a documented bound under a slow sink.
- Completion waits for all accepted records to settle.
- Route changes prevent obsolete results from reaching the new view.
Practice drill
Feed 47,000 small records through a slow sink and record peak queued bytes. Double the sink delay; memory should stay near the configured bound while completion time grows. Add one oversized record and verify explicit rejection. Switch routes halfway through and confirm old work stops without writing into the new view. Make record 29 fail and verify that the stream reports partial failure, not a completed feed. Compare sequential processing with bounded concurrency and document any order-repair mechanism.
Decision note
Let consumer readiness govern reads, and bound both item count and bytes wherever work is queued.
Common Mistakes
- Launching unawaited work for every streamed record.
- Bounding item count while ignoring byte size.
- Treating a dropped audit record as normal completion.
Connected lessons
Streaming and Large Data Interfaces; Incremental Response Framing and UTF-8; Stream Cancellation and Partial-Result Contract; Large Export Download and Integrity; Cooperative Main-Thread Scheduling; Workers, Messages, Transfer, and Cancellation; Browser Memory and Resource Lifecycles.
Apply and check
Build Project: streamed case activity and export and review Web Development: streams and interaction decisions quiz.
Further connections
Adaptive Media Buffer and Fallback Policy.
Further connections
web-tech
web-development