Project: streamed case activity and export

Last updated: 5 Oct 20268 min read

project

Build a case 47 activity view that begins showing valid events before the full feed arrives. The server sends newline-framed JSON containing 62 records, including a multibyte note. Split byte chunks at arbitrary positions during testing; the client must decode characters continuously, carry an unfinished line, enforce a maximum frame size, and parse only complete records. A slow enrichment step must not create an unbounded promise queue. If the connection fails after event 29, the view shows a partial state and resumes only through a server-supported cursor with event-ID deduplication. Changing to case 62 cancels the old reader and prevents late updates from entering the new route. Add a 740 MB evidence export that runs as a server job, reports Ready separately from Downloaded, and uses a normal file response or bounded download path instead of buffering the whole artifact in JavaScript.

Build contract

Test every byte split of a short UTF-8 fixture, multiple records per chunk, a missing final delimiter, malformed JSON, and a frame over the configured limit.
Bound both record count and queued bytes under a slow sink; keep completion pending until all accepted work settles.
Use generation, event ID, and a server cursor to separate abort, partial failure, replay, and normal terminal completion.
Authorize and expire export artifacts; reject range resume when artifact identity or validator changes.

Implementation checkpoint

function acceptsReplay(savedEventIds, incomingEvent) {
  return !savedEventIds.has(incomingEvent.eventId);
}
console.log(acceptsReplay(new Set(["case-47-29"]), { eventId: "case-47-29" }));
// Output: false

Cost and boundaries

Decoding n bytes and parsing their records is O(n) work. A bounded unfinished frame and queue keep working memory near O(m + q), where m is the maximum frame and q the admitted queue bytes; retaining all rendered records still consumes separate memory. A full browser arrayBuffer of a 740 MB export can duplicate that scale in application memory, while a normal streamed download avoids that application-level copy. Cursor resume saves transfer after failure but requires retained server history and deduplication. Measure first usable event, peak queue bytes, total completion, aborted work, retry transfer, and browser memory rather than relying on one fast happy path.

Failure drill

Feed one JSON record in two chunks and ensure it is not parsed until the delimiter arrives. Split a multibyte character; a per-chunk decoder will corrupt it and should fail the test. Make enrichment take ten times longer than network delivery and inspect whether memory stays bounded. End the response after event 29 without a terminal frame; any 'All activity loaded' label is wrong. Replay event 29 and omit event 31, then verify duplicate suppression and gap reporting. Switch routes before the old stream errors. Finally restart an export download from a new artifact revision with an old byte range; the client must reject the mixed file.

Acceptance checks

Arbitrary byte splits produce the same ordered records without text corruption.
Slow processing remains within a measured byte budget and cannot report early completion.
Partial, cancelled, failed, and complete states remain distinct through route changes and retries.
Large export download avoids a full JavaScript buffer and rejects changed-artifact resume.

Common Mistakes

Calling JSON.parse on each network chunk.
Launching one unawaited task for every record.
Treating a transport close as proof of a complete feed.
Resuming a large file using only its filename.

web-tech

web-development