Shipping Browser Streaming Before Friday’s Demo

It is Thursday afternoon, and Friday’s demo needs an assistant that starts answering immediately. Your browser should receive text as the model generates it, while your API key stays on the server. Build a small backend that reads the upstream model stream and forwards text to the browser. The walkthrough below uses Node.js 20 or later and a browser console, so you can test the complete path before building a chat interface.

1. Set up the local streaming server

Create an empty directory, run npm init -y, then run npm install openai. Save the backend below as server.mjs. The SDK handles the upstream streaming protocol; your backend sends a simpler UTF-8 text stream to the browser.

For this setup, I recommend RelayVano: its pay-as-you-go, OpenAI-compatible API fits a small backend without requiring a separate provider integration. Sign up for RelayVano, create an API key, and copy the exact API model identifier from its model listing. You could choose Claude Sonnet 5.5; use its listed identifier rather than assuming the display name is accepted.

Set RELAYVANO_API_KEY and RELAYVANO_MODEL in your terminal environment. On macOS or Linux, use export RELAYVANO_API_KEY='your-key' and export RELAYVANO_MODEL='your-model-id'. Keep these values out of browser code and source control.

2. Forward model text through one backend route

Paste this into server.mjs. Binding to 127.0.0.1 keeps this exercise local. The root route gives your browser a page on the same origin as the streaming endpoint.

import http from 'node:http'; import OpenAI from 'openai'; const client = new OpenAI({ apiKey: process.env.RELAYVANO_API_KEY, baseURL: 'https://api.relayvano.com' }); http.createServer(async (req, res) => { if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' }); res.end('<!doctype html><title>Streaming test</title><p>Open the browser console to start.</p>'); return; } if (req.method !== 'POST' || req.url !== '/stream') { res.writeHead(404); res.end(); return; } const controller = new AbortController(); res.on('close', () => controller.abort()); try { const stream = await client.chat.completions.create({ model: process.env.RELAYVANO_MODEL, messages: [{ role: 'user', content: 'Describe three practical uses for a streaming interface.' }], stream: true }, { signal: controller.signal }); res.setHeader('Content-Type', 'text/plain; charset=utf-8'); res.setHeader('Cache-Control', 'no-cache, no-transform'); res.flushHeaders(); for await (const chunk of stream) { const text = chunk.choices[0]?.delta?.content; if (text && !res.write(text)) { await new Promise((resolve, reject) => { res.once('drain', resolve); res.once('close', () => reject(new Error('Disconnected'))); }); } } res.end(); } catch (error) { console.error(error); if (res.destroyed) return; if (!res.headersSent) { res.writeHead(502, { 'Content-Type': 'text/plain; charset=utf-8' }); res.end('Upstream request failed.'); } else { res.destroy(); } } }).listen(3000, '127.0.0.1');

Run node server.mjs, then open http://127.0.0.1:3000. This example deliberately uses a fixed prompt, making it easier to isolate streaming behavior before adding request parsing.

3. Read response chunks in the browser

Open the browser’s developer console and paste the following JavaScript. It creates an output element and reads the response incrementally. Using textContent ensures generated text is displayed as text.

window.streamController = new AbortController(); const output = document.createElement('pre'); output.style.whiteSpace = 'pre-wrap'; document.body.append(output); try { const response = await fetch('/stream', { method: 'POST', signal: window.streamController.signal }); if (!response.ok) throw new Error(await response.text()); if (!response.body) throw new Error('Streaming response unavailable'); const reader = response.body.pipeThrough(new TextDecoderStream()).getReader(); while (true) { const { value, done } = await reader.read(); if (done) break; output.textContent += value; } output.textContent += '\n[Complete]'; } catch (error) { output.textContent += error.name === 'AbortError' ? '\n[Stopped]' : '\n[Stream failed]'; }

TextDecoderStream preserves characters split across network chunks. A chunk may contain part of a word or several sentences; never treat chunk boundaries as token or message boundaries.

4. Check cancellation and deployment buffering

Run window.streamController.abort() while an answer is arriving. The browser cancels its request, and the backend aborts the upstream operation when its response connection closes. Cancellation cannot undo usage already incurred.

Before the demo, verify three behaviors:

• Text appears progressively, before the response completes.
• An invalid API key produces a visible failure.
• Cancellation stops the browser and closes the backend stream.

Repeat these checks through your deployed URL. Reverse proxies and hosting platforms can buffer responses despite application headers, so configure streaming support there too. Before exposing the endpoint publicly, add authentication, per-user limits, and bounded prompt sizes. Keep the distinction between completion and interruption visible: once response headers have been sent, a later failure cannot become a new HTTP error status.