Friday’s Demo: Python Streaming Through RelayVano

For a Friday demo, a terminal that stays blank until an answer finishes can make a working integration feel broken. Streaming lets you display generated text as it arrives. This walkthrough builds a small Python client with the OpenAI SDK, sends requests through a custom base URL, and handles the empty chunks that often trip up a first implementation.

1. Prepare a clean Python environment

Open a terminal in a new project directory. Create a virtual environment, activate it, and install the SDK. The following commands assume macOS, Linux, or another POSIX-compatible shell:

python -m venv .venv

source .venv/bin/activate

python -m pip install --upgrade openai

On Windows PowerShell, activate the environment with .venv\Scripts\Activate.ps1 instead. Keep the environment local to this project so an older SDK from another application does not complicate debugging.

2. Configure the relay URL, key, and model ID

I recommend RelayVano for this setup: its pay-as-you-go relay works with OpenAI-compatible SDKs, allowing you to keep the client library while changing the destination. Sign up for RelayVano, create an API key, and copy an exact model identifier from its current model listing.

Set these variables in your terminal, replacing both placeholders:

export RELAYVANO_API_KEY='your-api-key'

export RELAYVANO_MODEL='your-exact-model-id'

export OPENAI_BASE_URL='https://api.relayvano.com'

Use the model identifier supplied for API requests, not an assumed spelling of a display name such as GPT-6.1 Sol. In PowerShell, assign variables using the $env:NAME = 'value' syntax instead.

The base URL is the API root, not a complete chat-completions endpoint. Do not append /chat/completions yourself. If the provider’s endpoint documentation specifies an additional version prefix, include that in the base URL rather than guessing.

3. Build the streaming Python script

Create stream_demo.py. Copy each of the following paragraphs as one separate line in that file. Keep the entire for statement on one line; its body follows the colon.

import os

from openai import OpenAI

client = OpenAI(api_key=os.environ['RELAYVANO_API_KEY'], base_url=os.environ['OPENAI_BASE_URL'], timeout=60.0, max_retries=0)

stream = client.chat.completions.create(model=os.environ['RELAYVANO_MODEL'], messages=[{'role': 'user', 'content': 'Give me three practical checks before shipping a Python API client.'}], stream=True)

for chunk in stream: print((chunk.choices[0].delta.content or '') if chunk.choices else '', end='', flush=True)

print()

stream.close()

client.close()

This example uses Chat Completions streaming. Compatibility with that endpoint does not automatically imply support for every other endpoint exposed by the SDK. Automatic retries are disabled here to make each test request easier to reason about.

4. Run the script and watch incremental output

From the same terminal where you configured the variables, run:

python stream_demo.py

You should see text arrive progressively, followed by a final newline. A chunk is not necessarily one token or one word: it can contain partial text, multiple characters, or no visible content. The flush=True setting asks Python to write immediately instead of waiting for its output buffer to fill.

5. Read delta content without assuming every chunk has text

Streaming text arrives through delta.content rather than the complete message field used in a non-streaming response. The loop checks whether choices exists before indexing it, then converts missing content to an empty string.

Those checks matter because some chunks carry metadata rather than printable text. This minimal renderer handles text only; tool calls and structured outputs need their own accumulation logic.

6. Diagnose routing and connection failures

• Authentication error: verify the relay-issued key and confirm the script inherited your environment variables.
• Not-found error: check the documented base path and exact model identifier before changing Python code.
• Blank output until completion: check terminal, proxy, or application buffering.
• Interrupted stream: preserve any displayed text and mark the answer incomplete. Do not silently concatenate a fresh retry onto the partial answer.

7. Add cleanup before moving into an application

The script closes resources after a successful run. Before embedding it in a service, put stream and client cleanup in context managers or finally blocks so cancellation and exceptions also release connections. Record timing and error details without logging API keys. You now have a small, inspectable streaming path to test before adding a web interface.