Incremental Response Framing and UTF-8

Working case

The case 47 activity feed sends 62 records as newline-delimited JSON. A network chunk ends halfway through the word 'reviewed', and another splits a Hindi note between UTF-8 bytes. A parser that runs JSON.parse on each chunk fails unpredictably or replaces characters. Decode bytes across chunk boundaries, append text to a bounded tail, and emit records only after a newline. A malformed line should carry its record position into an error state rather than silently disappearing. The interface may show accepted earlier records, but it must not call the whole feed complete if later parsing fails.

Implementation boundary

function splitCompleteLines(previousTail, textChunk) {
  const pieces = (previousTail + textChunk).split("\n");
  return { lines: pieces.slice(0, -1), tail: pieces.at(-1) };
}
const first = splitCompleteLines("", "{\"caseId\":47");
const second = splitCompleteLines(first.tail, "}\n{\"caseId\":62}\n");
console.log(second.lines.length, second.tail.length);
// Output: 2 0

The helper models text framing after a TextDecoderStream or stateful TextDecoder has handled partial UTF-8 bytes. The production reader should check response.ok, acquire the stream once, pass decoded text through the splitter, parse complete lines with try/catch, and enforce a maximum tail size before appending unbounded input. Record sequence numbers help detect a missing frame even when every received line parses. Decide whether to parse a nonempty tail at end-of-stream according to the server protocol, and close or cancel the reader when the route no longer owns it.

Cost and boundaries

For n total bytes, decoding and framing are O(n) time, with O(m) working memory for the largest permitted incomplete frame plus retained results. Calling response.text() first also costs O(n) memory for the full body. Repeated string concatenation of a large unbounded tail can become expensive, which is another reason to cap frame length. JSON parsing remains proportional to each record's size. Streaming helps first-result latency, but rendering every record immediately can still block the main thread; batch UI commits or move heavy transforms to a worker when measurements justify it.

Failure trace

An implementation assumes each read() result is one JSON object. Tests use small local responses and pass, but production buffering splits a single record into two reads and the page reports a parse error. Another implementation decodes each byte chunk independently and corrupts a multibyte character at the split. Treat transport chunks as arbitrary, use a streaming text decoder, and test every possible split in a short fixture. If the server closes after half a record, expose a partial-feed failure and keep the last confirmed sequence visible.

Verification

• Every byte split yields the same two parsed records.
• An incomplete final frame has a documented outcome.
• A malformed or oversized record cannot become silent success.

Practice drill

Construct a two-record feed and split its bytes after every possible offset, including inside a multibyte note. Confirm that all splits produce the same records. Send two records in one chunk and one record across three chunks. Omit the last newline and compare behavior under the chosen protocol. Grow one line past its limit and verify the reader stops with a bounded error. Insert malformed JSON at record 29 and confirm the UI labels prior records as partial rather than complete.

Decision note

Treat chunks as arbitrary bytes; decode continuously, frame explicitly, and bound the unfinished record.

Common Mistakes

• Parsing each network chunk as one complete record.
• Decoding UTF-8 bytes independently at each chunk.
• Allowing an unfinished line to grow without a limit.

Connected lessons

Streaming and Large Data Interfaces; Stream Backpressure and Bounded Work; Stream Cancellation and Partial-Result Contract; Large Export Download and Integrity; Fetch requests: separate HTTP failure, transport failure, and cancellation; Server-Sent Events or WebSocket for Live Views; Workers, Messages, Transfer, and Cancellation.

Apply and check

Build Project: streamed case activity and export and review Web Development: streams and interaction decisions quiz.

Further connections

Generated Response Streaming and Cancel State.