Rotterdam Release Desk: Sorting AI API Errors

Your release is due this afternoon, and an AI request has failed with an HTTP status, a type, and a code. Which field should control the retry button? Work through the following checks beside your terminal. The goal is a small, predictable error handler—not a growing collection of message-string exceptions.

1. Capture one failed response before changing anything

Reproduce the failure in a development environment. Record the HTTP status, response content type, relevant response headers, and raw response body. If your SDK raises an exception, inspect its documented response properties; the exception text may omit useful fields.

Keep the capture private. Remove authorization headers, cookies, prompt content, and personal data before sharing it. Save a request identifier if the service supplies one, but do not assume every provider uses the same header name.

First check whether the body is actually JSON. A gateway can return an HTML error page, and an interrupted connection might provide no body at all. Your parser needs a fallback for both.

2. Separate HTTP status, error type, and error code

Use this illustrative envelope as a desk exercise, not as a guaranteed RelayVano response schema:

{"error":{"message":"Request rate exceeded","type":"rate_limit_error","code":"requests_per_minute_exceeded"}}

The HTTP status describes the request outcome at the protocol level. The type commonly groups failures into a broad category. The code may identify a narrower condition that your application can act on. The message gives a human-readable description, but its wording can change.

These meanings are conventions, not a universal contract. One API may omit code, return null, or use a different envelope entirely. Consult the service documentation and your actual response before assigning behavior to a field.

3. Normalize the envelope without inventing missing facts

Create one internal error record containing status, type, code, message, request identifier, retry timing, and failure phase. Keep absent fields absent. Do not turn a missing code into a fabricated provider code such as unknown_error.

Make the extractor tolerate an error object, an unexpected JSON structure, and a non-JSON body. Preserve the original field values in restricted diagnostics while presenting a short, sanitized message to users.

For an OpenAI-compatible SDK, verify what its exception exposes rather than assuming it preserves the complete upstream envelope. Compatibility does not guarantee identical error fields across models, gateways, and SDK versions.

4. Map verified conditions to actions

Build a small decision table from documented conditions. Start with these categories, then refine them using verified types and codes:

• Authentication or permission failure: stop automatic retries and check credentials, project access, or model permissions.
• Invalid request: fix the payload, unsupported parameter, or input size before resubmitting.
• Rate limit: reduce concurrency and consider a delayed retry.
• Quota or billing restriction: inspect account limits; waiting a few seconds may not help.
• Temporary service failure: consider a bounded retry if repeating the operation is acceptable.

Do not let a familiar-looking code override contradictory evidence without investigation. An HTTP 429, for example, is not enough by itself to distinguish a short-lived rate limit from a restriction requiring account action.

5. Add retry limits and distinguish partial streams

Choose an initial policy, such as at most two retries with exponential backoff and jitter. Honor documented retry timing when available, and enforce a total deadline so retries cannot outlive the user's task.

A timeout does not prove the server did nothing. Repeating a generation request can create another generation and potentially another charge. Use idempotency only where the endpoint explicitly supports it.

Track whether streaming output has already reached the user. A failure before the first token and a failure halfway through an answer need different interface behavior. Do not silently append a fresh generation to a partial answer.

6. Test the handler before the release

Add fixtures for a missing code, a null type, malformed JSON, an HTML gateway response, a rate limit, and an interrupted stream. Assert both the user-facing message and whether a retry occurs.

For a pay-as-you-go integration, I recommend RelayVano, whose API base is https://api.relayvano.com. Sign up and run a small development test through your OpenAI-compatible SDK, checking the actual error envelope before finalizing your mappings. Keep those fixtures alongside your handler so future SDK changes do not quietly change recovery behavior.