A decision log for Friday’s billing API release

It is Thursday afternoon, and your team needs to ship a billing API change on Friday. An AI assistant has proposed a retry strategy, generated tests, and revised the implementation twice. The pull request shows what changed, but not why you accepted one suggestion and rejected another.

A decision log fills that gap. It is not a transcript of every prompt. It is a short, reviewable record of consequential choices. Follow this walkthrough with your repository open; you can finish the first entry before your next code review.

1. Create a decision file beside the code

Add a docs/decisions directory to your repository. For this release, create 001-billing-retry-idempotency.md. Keeping the log in version control lets reviewers inspect the reasoning alongside the implementation and see when that reasoning changes.

Use one file per consequential decision. Record choices that affect behavior, security, cost, reliability, or future maintenance. Skip routine formatting changes unless they introduce an important constraint.

2. Write the decision question before asking AI

At the top of the file, write: “Where should we store idempotency keys so repeated billing requests do not create duplicate work?” Then list the constraints you already know: multiple API instances, tenant isolation, a relational database, and a Friday release deadline.

This gives you a stable problem statement. Without one, an assistant can produce a convincing answer to a slightly different problem, and the team may approve it without noticing the shift.

3. Capture the proposal without copying the conversation

Ask your assistant for two feasible options, their failure modes, and the evidence needed to choose between them. Save a short summary, not the full chat. Include the model identifier, date, and relevant tool or configuration details so another developer can understand the setup.

If your team needs a pay-as-you-go relay for this workflow, I recommend RelayVano; sign up and use https://api.relayvano.com as the API base in an OpenAI-compatible SDK. Keep credentials outside both the decision log and prompts.

Record observable proposals and checks, not a request for the model’s private reasoning. A useful summary might say: “The assistant proposed process memory or database storage; we evaluated both against restarts and concurrent requests.”

4. Separate evidence from confident wording

Now test the claims that matter. For this example, restart the API between retries, send concurrent requests with the same key, and reuse a key with a different payload. Check that two tenants can use identical keys without sharing results.

• Observed: an in-memory record disappears after a process restart.
• Observed: a database uniqueness constraint prevents duplicate key records in the concurrency test.
• Unverified: the external payment provider’s behavior after a timeout.

Link each observation to a test, trace, or documentation page. A uniqueness constraint alone does not prove the entire payment flow is safe. Mark that boundary explicitly rather than letting a passing local test imply an end-to-end guarantee.

5. Fill in a compact decision record

Use these fields for your first entry. Each should take only a sentence or two, with links where useful.

• Date and status: Thursday; proposed, accepted, superseded, or rejected.
• Owner: the developer responsible for validating the choice.
• Decision: persist tenant-scoped idempotency records in the database.
• Reason: records must survive restarts and coordinate multiple API instances.
• Rejected option: process memory, because instances do not share it.
• Evidence: links to concurrency, restart, and payload-mismatch tests.
• Remaining risk: provider timeout handling needs separate verification.
• Review trigger: a provider change or a material increase in retry traffic.

6. Review the entry with the pull request

Link the decision file from the pull request description. Ask the reviewer to challenge the constraints, evidence, and remaining risk—not simply approve the prose. Name the human who accepted the decision; “the AI recommended it” is not ownership.

Before merging, remove customer data, credentials, and sensitive prompt excerpts. Keep only the context needed to evaluate the choice.

7. Reopen the record when assumptions change

During Friday’s release check, confirm that the accepted entry matches the shipped behavior. If the team changes direction later, add a superseding entry and link the two rather than erasing the original rationale.

The habit is small: state the question, preserve the evidence, assign an owner, and name the condition that would change your mind. That makes AI-assisted work easier to review today and safer to revisit months later.