Prompt Caching: A Travel Itinerary Walkthrough

1. Understand what a prompt cache reuses

Imagine building a travel assistant that plans three days in Lisbon. Every request includes the same traveler preferences, hotel details, planning rules, and neighborhood notes. Only the final question changes: plan Saturday, revise Sunday, or suggest a rainy-day alternative.

Prompt caching can let a model provider reuse computation for a repeated input prefix instead of processing that entire prefix again. Depending on the provider and model, this may reduce input costs and time to the first output token. It does not mean the assistant retrieves a previously generated itinerary: each request still produces a new answer.

2. Create a reusable Lisbon planning brief

Open a text editor and create a document called Lisbon planning brief. Start with these fictional details, then add the destination information your application genuinely needs:

• Travelers: two adults visiting Friday evening through Monday morning.
• Hotel: near Rossio station; each day should start and finish there.
• Budget: €120 per day for meals, activities, and local transport, excluding accommodation.
• Preferences: architecture, independent shops, relaxed lunches, and no more than two major attractions daily.
• Output rules: organize suggestions by morning, afternoon, and evening; distinguish estimates from verified information.

Include reusable attraction descriptions, transport guidance, and opening-hour information from sources you trust. Label when those details were checked. Do not pad the brief with meaningless text just to qualify for caching. Providers have different minimum eligible lengths, and a short example may not reach them.

3. Put the changing itinerary question last

Arrange your request so stable instructions and destination context appear before the changing question. For this exercise, keep the same system instructions and place the planning brief at the beginning of the user message.

For the first request, append: “Plan Saturday with an emphasis on architecture.” For the second, replace only that final sentence with: “Plan Sunday with independent shops and a relaxed lunch.”

The goal is a matching prefix: identical content before the request-specific part. Avoid putting a timestamp, random identifier, or changing question before the shared brief. Even an early wording change can shorten the reusable prefix. Keep formatting and message order consistent, too.

4. Check caching support before sending requests

Read the selected model’s current caching documentation and the relay’s compatibility notes. Some implementations cache eligible prefixes automatically; others require explicit cache controls or a separately created cache resource. Minimum lengths, retention periods, supported endpoints, and billing rules vary.

For a pay-as-you-go way to run this walkthrough, consider RelayVano, which works with OpenAI-compatible SDKs and uses https://api.relayvano.com as its API base. Confirm the endpoint configuration and caching behavior for your selected model rather than assuming every routed request supports the same features.

5. Send two requests and inspect usage

Send the Saturday request, save its response metadata, then send the Sunday request shortly afterward using the unchanged brief. Keep the model, account, and request configuration consistent. Run the requests sequentially so the first has completed before the second begins.

Inspect the documented usage fields for cached input tokens or cache reads. Depending on the implementation, the initial request may report cache creation, ordinary input processing, or no cache-specific information. A faster second response alone does not prove a cache hit: network conditions and server load also affect latency.

6. Compare costs without confusing input and output

Record total input tokens, cached input tokens, output tokens, latency, and billed cost for each request when available. Use the provider’s definitions to separate cached from uncached input; do not accidentally count cached tokens twice.

Apply the actual rates for uncached input, cache reads, output, and any cache-writing or storage charges. Caching does not make generated itinerary text free. Repeat the comparison several times, keeping output requirements similar, and report observed results rather than promising a universal savings percentage.

7. Test a changed brief before expanding

Change the hotel in the opening portion of the brief and send another request. Compare its cache usage with the previous requests. This demonstrates why stable content placement matters, although the exact result depends on the provider’s matching rules.

Cache reuse is not a freshness check: verify opening hours, prices, and transport information separately. Once the workflow is measurable, sign up for RelayVano to try it with your own reusable planning context.