Guide
Counting tokens and estimating the cost of LLM prompts
Language models read and write text in tokens, and everything about using them is measured in tokens: how much fits in the context window, how long a response may be and what each call costs. Counting words or characters only approximates that. This counter runs the actual tokenizer used by OpenAI models on your text, shows where the token boundaries fall and turns the count into a cost estimate.
What a token is
Models do not see letters or words but pieces of text from a fixed vocabulary, built with byte-pair encoding: frequent sequences such as common words or word endings become single tokens, and rare words are split into several. “ shipping” with its leading space might be one token, while an unusual name breaks into three or four. Spaces, punctuation and line breaks belong to tokens too, which the highlighted view makes visible.
The two encodings
The counter implements the two byte-pair encodings OpenAI’s current models use. o200k_base, with a vocabulary of about 200,000 tokens, is used by GPT-4o and newer models. cl100k_base, with about 100,000, is used by GPT-4 and GPT-3.5 Turbo. The larger vocabulary usually needs fewer tokens for the same text, especially for code and non-English languages. For models that use these encodings, the count is exact.
Context windows
A model’s context window is the total number of tokens it can consider at once, including the system prompt, the conversation so far, any documents you include and the response it writes. Pick a window size to see how much of it the text fills; the meter turns red and a warning appears when the text is longer than the window, a sign to split the document, summarise parts of it or retrieve only relevant sections.
Estimating cost
API pricing is quoted per million tokens, with different prices for input (what you send) and output (what the model writes). Enter the prices from your provider’s pricing page, the number of tokens you expect in each response and how many calls you make per day. The estimate is (input tokens × input price + output tokens × output price) ÷ 1,000,000 per call, then per day and per 30 days. Prices change and providers offer discounts for cached input and batch requests, so no prices are built in.
A worked example
The Sample is a short support-assistant prompt with a customer message. It is 297 characters long and counts 70 tokens with o200k_base, about 4.2 characters per token. Switching to cl100k_base gives 73 tokens for the same text. With the default figures of 2.5 and 10 dollars per million input and output tokens and 500 expected output tokens, each call costs about half a cent, and 1,000 calls a day come to about $155 over 30 days.
Other model families
Claude, Gemini, Llama, Mistral and other model families each have their own tokenizer and vocabulary, so the same text gives different counts. The number here is a reasonable approximation for them, often within a modest margin for English, but use the provider’s own token-counting endpoint or library when exact numbers matter for billing or limits. Chat APIs also add a few tokens per message for role markers.
Privacy
The tokenizer runs in your browser; the text you paste is not sent to any AI provider or to this site. That makes it safe to measure confidential prompts and documents before deciding what to send.
Related tools
- Prompt Template Filler builds many prompts from one template.
- Word Counter gives words, sentences and reading time.
- Character Counter counts characters against platform limits.
- JSON Lines Converter prepares batch and fine-tuning files.
Checked against the current version of the LLM Token Counter on .