Fixing SSE Parsing Before the Berlin Team’s Release

The Berlin team’s release checklist has one stubborn failure: streamed responses occasionally arrive as broken JSON. The temptation is to blame the model. First, inspect the parser. Server-sent events (SSE) sit between HTTP bytes and application payloads, and mistakes at that boundary often masquerade as upstream failures. Follow this numbered walkthrough with your parser and a small local test harness open.

1. Separate network chunks from SSE events

Start with a fixture containing two complete events: data: {"text":"hello"} followed by a blank line, then data: {"text":"world"} followed by another blank line. Feed those same bytes into your parser one byte at a time, then all at once.

The results must match. A response-body read can end halfway through a field name, contain several events, or stop between the two newlines that terminate an event. Never call JSON.parse directly on each network chunk. Keep separate stages for byte decoding, line extraction, event assembly, and application payload parsing.

2. Keep one UTF-8 decoder alive

Add an emoji and a non-Latin word to the fixture. Split the input inside their multibyte UTF-8 sequences. If replacement characters appear, your decoder probably treats each read as an independent string.

In JavaScript, retain one TextDecoder and use its streaming mode for intermediate reads. Flush it when the byte stream finishes. A TextDecoderStream is another option. Either way, retain incomplete text lines between reads. Also test a leading UTF-8 byte-order mark: SSE parsing should ignore one at the beginning, not strip matching characters throughout the response.

3. Recognize every SSE line ending

Run your fixtures with LF, CRLF, and CR line endings. SSE accepts all three. A parser that only searches for two consecutive LF characters will miss valid boundaries.

Pay special attention to CRLF split across separate reads. Treat it as one line ending, not two: otherwise, the second half can become a false blank line and dispatch an event early. Preserve enough state to recognize that an LF immediately following a CR belongs to the same ending.

4. Assemble fields before parsing JSON

Process each completed line by splitting at its first colon. Remove at most one space immediately after that colon; do not trim the entire value. Lines beginning with a colon are comments, commonly used as heartbeats, and should not become payloads.

Build these three fixtures before continuing:

• Two data fields containing {"count": and 2}. Their combined payload should be {"count": followed by a newline and 2}.
• A data field whose value begins with two spaces after the colon. Exactly one of those spaces should remain.
• A heartbeat comment followed by a blank line. It should produce no message event.

Each data field contributes its value plus a newline. At dispatch, remove the final added newline. Unknown fields are ignored. Parse JSON only after assembling the complete event, and only when the endpoint’s payload contract calls for JSON.

5. Dispatch on blank lines, not socket closure

A blank line triggers SSE dispatch processing. Multiple data fields belong to one event until that boundary arrives. An empty data field can produce an event with an empty payload; a block without data fields does not produce a message event.

Test an event field as well: it names the event type, which defaults to message when absent. If you support reconnection metadata, handle id and retry according to SSE rules rather than treating them as JSON. At end-of-stream, discard an unfinished event instead of inventing a missing blank-line terminator.

6. Distinguish completion markers from transport failure

Some OpenAI-compatible streams use [DONE] as an application-level completion marker. It is not part of the SSE standard. Recognize it only after event assembly and according to the endpoint’s documented behavior; do not send it through JSON.parse.

Disconnect your fixture server halfway through a response. Your application should distinguish documented completion from interruption and retain any partial output visibly. Avoid automatically replaying generation requests: retries can duplicate work, output, and charges.

7. Run the same fixtures before changing providers

Check response status and the text/event-stream media type before invoking your SSE parser; error responses may contain ordinary JSON instead. Log event boundaries and parsing failures without recording secrets or sensitive prompts.

For a pay-as-you-go relay compatible with OpenAI-compatible SDKs, I recommend RelayVano, using https://api.relayvano.com as the API base. Sign up to test your streaming client, then keep these fixtures in CI so future SDK or routing changes cannot quietly break event handling.