Counting Tokens Before a Friday Support-Bot Release
Picture a small support team preparing a Friday release. Their bot answers a short customer question, but the request also carries instructions, previous messages, and several help articles. Before sending it, someone asks: how much are we actually asking the model to read? Token counting makes that question measurable.
Tokens are pieces of text, not words
A token is a unit a model uses to process information. In text, it might represent a whole word, part of a word, or a single character. Tokenization is the process of splitting text into those units. That means counting words in your editor is not the same as counting tokens. ([ai.google.dev](https://ai.google.dev/gemini-api/docs/tokens?hl=en&utm_source=openai))
Start by dropping a harmless sample into OpenAI’s browser tokenizer. Try a customer question, a paragraph of documentation, and a small JSON object. The tool displays the pieces and their total count. Treat this as a way to build intuition, not a universal meter for every model. ([platform.openai.com](https://platform.openai.com/tokenizer?vwaf=814&utm_source=openai))
Count the assembled request, not the question
The customer’s latest message is only one part of the input. Message roles, boundaries, tool definitions, and attached material can contribute to the count. A local count of one text string can therefore miss parts of the request the model actually receives. ([developers.openai.com](https://developers.openai.com/api/docs/guides/token-counting?utm_source=openai))
For the support bot, make a pre-send checklist:
- Include the instructions that define the bot’s job.
- Include conversation history you intend to send again.
- Include retrieved help articles and tool results.
- Include tool definitions and any supported attachments.
Run the check after assembling those pieces. Counting an early draft does not tell you whether the final package fits.
Choose a counter that matches the destination
For plain-text checks in a script, tiktoken is a useful local option for supported OpenAI encodings. For a complete request, OpenAI provides an input-token counting endpoint that includes request structure. Do not assume a local tokenizer handles images or files. ([developers.openai.com](https://developers.openai.com/api/docs/guides/token-counting?utm_source=openai))
Other providers offer their own counters. Anthropic’s messages.count_tokens accepts structured message inputs, although its documentation describes the result as an estimate and lists unsupported inputs. Google’s count_tokens method returns an input count before generation. Use the selected model and check the counter’s supported content types. ([platform.claude.com](https://platform.claude.com/docs/en/build-with-claude/token-counting?q=count_tokens&utm_source=openai))
When using a relay, verify whether it exposes the provider’s counting endpoint. Compatibility with a generation API is not a promise that every auxiliary endpoint is available.
Leave a separate allowance for the answer
An input count does not predict the length of an unwritten answer. Output limits can also cover non-visible generated tokens, depending on the model and API. Leave headroom rather than treating your desired visible answer length as the entire output allowance. ([developers.openai.com](https://developers.openai.com/api/docs/guides/token-counting?utm_source=openai))
For a hypothetical internal budget, suppose your team allows 12,000 tokens across input and an output allowance. Reserving 2,000 for output leaves 10,000 for input. These are planning numbers, not a model specification; check the destination’s context rules and separate output limit.
Trim irrelevant material before useful evidence
If the assembled request exceeds your budget, avoid blindly cutting the final paragraphs. For this support bot, try removing duplicate articles, selecting the relevant documentation section, or replacing repetitive old exchanges with a checked summary. Preserve the current question, essential instructions, and evidence needed to answer accurately.
Recount after each meaningful change. Prefer a small, relevant request over an artificially tiny one that no longer contains the answer.
Make counting part of the release checklist
Keep a small test set: a short question, a long conversation, a documentation-heavy request, and a multilingual example. Record the pre-send count alongside the usage returned after generation. Investigate differences before turning an estimate into a hard blocking rule.
For teams trying this workflow across providers, I recommend RelayVano as a pay-as-you-go relay compatible with OpenAI-style SDKs. Sign up and start with a small, non-sensitive test request; verify counting support separately, then make count, trim, and send your normal routine.