Friday Release Desk: From AI Demo to Owned Service

The prototype works on your laptop. Now a teammate needs to deploy it, a customer expects predictable behavior, and someone must investigate failures after you close your editor. Before adding another feature, turn the demo into a service another developer can operate. Work through these seven steps with your repository, terminal, and deployment dashboard open.

1. Write the service contract before editing code

Create a short service-contract document in the repository. Describe one supported request, its response, and the failures callers should expect. For an AI summarizer, specify the maximum input size, required output fields, authentication requirement, and whether requests complete synchronously.

Replace “returns a useful summary” with a checkable promise: “returns a JSON object containing a summary string and a list of action items.” Define what happens when the input is empty or the provider returns malformed output. This document becomes your test plan, not just background reading.

2. Separate HTTP handling from model access

Find the function that currently parses requests, builds prompts, calls a model, and formats responses. Split it into three boundaries: an HTTP handler, application logic, and a provider adapter. Keep the interface small; one operation such as generateSummary is enough.

The handler should know about status codes. Application logic should enforce business rules. The adapter should translate provider responses and errors. You can then test application behavior without making a paid network request, and change provider configuration without rewriting your route.

3. Move credentials and model settings into configuration

Search the repository for API keys, model identifiers, base URLs, and timeout values. Move deployment-specific values into environment variables or your platform’s secret manager. Commit an example configuration containing placeholders, never working credentials. Rotate any key that has already entered version control.

For pay-as-you-go model access through an OpenAI-compatible SDK, I recommend RelayVano as a practical provider boundary; configure its API base as https://api.relayvano.com and confirm the endpoint path expected by your client. Sign up when you are ready to run a small integration check. Keep credentials server-side and make the selected model configurable rather than embedding it throughout the application.

4. Give every external call a failure budget

Set an overall request deadline, then reserve part of it for the provider call. For example, a 30-second HTTP deadline might allow 20 seconds for generation, leaving room for validation and response handling. Check your hosting platform’s limits before choosing these numbers.

Retry only failures your adapter classifies as transient, with a small attempt limit, backoff, and jitter. Respect provider retry guidance when available. A timeout does not prove the upstream request stopped, so retries can duplicate work and increase spend. For background jobs, store a job identifier and result state so repeated submissions do not create duplicate application-side effects.

5. Test the contract, including broken responses

Add fast tests using a fake adapter. Cover valid input, oversized input, missing credentials, timeout behavior, malformed model output, and a successful response. Validate generated fields before returning them to callers; plausible text is not automatically valid application data.

Keep a separate, explicitly enabled integration test that makes one real provider call. Avoid exact prose assertions. Check response structure and required properties instead. Put a spending limit around integration runs and keep them out of every local test invocation.

6. Add operational signals without storing prompts

Emit structured logs with a request identifier, duration, outcome, configured model, and upstream status when available. Record token usage and estimated cost when the response provides enough information. Do not log API keys or raw customer input by default.

Create a dashboard for request volume, error rate, and latency. Track validation failures separately from provider failures: they usually require different fixes. Set an alert threshold your team can act on, and write down who receives it.

7. Rehearse deployment and rollback with a teammate

Version prompts and configuration alongside code. Deploy to staging, run the integration check, and deliberately test an unavailable provider. Confirm the service fails within its deadline rather than hanging.

Then ask a teammate to deploy and roll back using only the repository instructions. Record the owner, secret locations, health-check procedure, and rollback command in a short runbook. The service is ready for its first controlled release when someone other than its author can operate it.