LLM Basics: Tokens, Context Windows, and Cost Math

The word "token" on your first LLM API bill can be confusing: is it a word, a character, or something else? What a token is, how it determines the context-window cost math, and what happens when the window fills up — this article answers all three from a single foundation, grounded in providers' official pricing and documentation pages. If you're looking for a model-agnostic starting point, you're in the right place.

💡 Pro Tip: When estimating a bill, don't look only at output tokens — look at the ENTIRE input, including the system prompt, message history, and tool definitions. Most surprise bills come from forgotten "silent" input tokens.

Table of Contents

• What a Token Is and What a Tokenizer Does
• Why Token Cost Is Higher in Non-English Languages
• Context Window: Input + Output + Reasoning
• What Happens When the Window Fills Up
• Billing Math with Input/Output/Cache Prices
• A Simple Cost Estimation Table
• 7 Ways to Cut Your Token Budget
• FAQ
• What is a token, and how many tokens does a word cost?
• What happens when the context window fills up?
• How is LLM cost calculated?
• Why does text in other languages cost more tokens than English?
• How does cache pricing work?
• When does the Batch API make sense?
• Update (September 2026)
• Conclusion
• Sources

What a Token Is and What a Tokenizer Does

An LLM doesn't process text letter by letter; it processes it in sub-word chunks called "tokens." According to OpenAI's official explanation, a token can roughly be a piece of a word; in English, about 4 characters or 0.75 words correspond to one token (help.openai.com, 2025-01-19 archive). That translates into a rough rule of thumb like "100 tokens ≈ 75 words" — but this ratio varies significantly by language.

A tokenizer is the component that splits raw text into these sub-word units. As the model processes each token, it computes the probability distribution for the next token; so both input and output tokens are the units the model actually "sees" and produces — not character counts.

1# Rough English token estimate (OpenAI's "~4 characters = 1 token" rule)
2import math
3 
4def estimate_tokens_en(text: str) -> int:
5    return max(1, math.ceil(len(text) / 4))
6 
7print(estimate_tokens_en("How are you today?"))  # 5 tokens (18 characters / 4 = 4.5 → round up)

Real tokenization is more complex than this (it uses byte-pair-encoding-based sub-word vocabularies) — the function above is only a rough estimation tool; an exact count requires the provider's tokenizer library.

Why Token Cost Is Higher in Non-English Languages

OpenAI's tokenization documentation illustrates with an example that tokenization varies by language: "'Cómo estás' ('How are you' in Spanish) contains 5 tokens (for 10 chars)" — meaning the token-to-character ratio can come out higher in non-English languages (help.openai.com, 2025-01-19 archive). In the documentation's own words, this higher token-to-character ratio can make the API more expensive to use for non-English languages (help.openai.com, 2025-01-19 archive).

The current Help Center page preserves the same structural point in general form: the relationship between characters, words, and tokens can vary from language to language (help.openai.com). The practical takeaway: when budgeting for non-English input/output, you need to leave extra headroom compared to English.

Context Window: Input + Output + Reasoning

The context window is the total number of tokens the model can "remember" within a single request — the system prompt, chat history, attached files, tool definitions (input), and the model's generated response (output) all draw from this window. As of 2025-01-21, OpenAI's o1 model offered a 200,000-token context, while the standard GPT-4o model had a 128,000-token context window (https://web.archive.org/web/20250121103038/https://openai.com/api/pricing/). In the same period, Claude 3.5 Sonnet and Claude 3.5 Haiku operated with a 200k-token context window (https://web.archive.org/web/20250119203513/https://docs.anthropic.com/en/docs/about-claude/models).

"Reasoning" models, e.g. OpenAI's o1 series, run an internal "thinking" phase before producing the response, and the reasoning tokens generated by that phase are also deducted from the output token budget — and therefore from the bill — even if they aren't shown to the user. This means two requests with the same visible response length can produce very different costs when one runs on a reasoning model and the other on a classic model.

1Context window = system prompt + message history + attached file/tool output
2                + user message (input tokens)
3                + model response + (if any) reasoning tokens (output tokens)

What Happens When the Window Fills Up

When the context window fills up or is exceeded, the request limit has been exceeded; OpenAI's support documentation recommends condensing the prompt or splitting the text into smaller chunks in this case (help.openai.com, 2025-01-19 archive). In practice there are three common strategies:

• Truncation: Drop the oldest messages from the window and send only the most recent N messages.
• Summarization: Replace old history with a short summary, freeing tokens back into the window.
• Chunking: Process a large document across successive small requests instead of a single request.

Whichever strategy you choose, the outcome rests on the same principle: the window is a fixed resource, and every token you fill it with becomes unavailable for the next message.

Billing Math with Input/Output/Cache Prices

Providers price input and output tokens differently — output tokens are almost always more expensive than input tokens, because generating them requires the model's active computation step.

As of 2025-01-21, OpenAI's official pricing page (https://web.archive.org/web/20250121103038/https://openai.com/api/pricing/) listed the following figures: for GPT-4o, $2.50 input / $10.00 output per 1 million tokens, $1.25 for cached input; for GPT-4o mini, $0.150 input / $0.600 output; for the o1 reasoning model, $15.00 input / $60.00 output. In the same period, on the Anthropic side, Claude 3.5 Sonnet was listed at $3.00 input / $15.00 output, Claude 3.5 Haiku at $0.80 input / $4.00 output, and Claude 3 Opus at $15.00 input / $75.00 output (https://web.archive.org/web/20250119203513/https://docs.anthropic.com/en/docs/about-claude/models).

During this period, both providers also offered a discount for batch processing — OpenAI's Batch API ran at roughly half the standard price (a 50% discount) with a 24-hour SLA (https://web.archive.org/web/20250121103038/https://openai.com/api/pricing/).

1{
2  "model": "gpt-4o",
3  "usage": {
4    "prompt_tokens": 1200,
5    "completion_tokens": 340,
6    "total_tokens": 1540
7  }
8}

Billing math is simply multiplying the prompt_tokens (input) and completion_tokens (output) values in this usage object by the relevant unit price and summing them.

A Simple Cost Estimation Table

The table below summarizes the official list prices (USD per 1 million tokens) that were in effect in the 2025-01-21 period — always check the provider's live pricing page for current figures.

Example calculation: a GPT-4o request with 1,200 input + 340 output tokens costs (1200/1,000,000 × 2.50) + (340/1,000,000 × 10.00) = 0.0030 + 0.0034 = $0.0064. That looks small for a single request, but for a production system running 10,000 similar requests per day, that's ~$64/day.

Comparing the same 1,200/340 token profile across other models shows how much model choice affects the bill. GPT-4o mini: (1200/1,000,000 × 0.150) + (340/1,000,000 × 0.600) = 0.00018 + 0.000204 = $0.000384 — roughly one-seventeenth of GPT-4o. Claude 3.5 Sonnet: (1200/1,000,000 × 3.00) + (340/1,000,000 × 15.00) = 0.0036 + 0.0051 = $0.0087. OpenAI o1: (1200/1,000,000 × 15.00) + (340/1,000,000 × 60.00) = 0.018 + 0.0204 = $0.0384 — this covers only the visible 340 output tokens; a reasoning model also bills unseen reasoning tokens at the same output price. Model choice alone can shift the bill by up to ~100x here; that's why "pick the right model first" comes before optimizations like caching or batching.

For a production system with daily volume, rather than redoing this math by hand repeatedly, it's more reliable to write the usage field from the API response directly into a log/aggregation pipeline and sum it there — bill estimation should be based on real usage data, not an estimated average.

1# Fetch the usage field from the API response and append it to a daily log file (example)
2curl -s https://api.openai.com/v1/chat/completions \
3  -H "Authorization: Bearer $OPENAI_API_KEY" \
4  -H "Content-Type: application/json" \
5  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"merhaba"}]}' \
6  | jq '.usage' >> usage-log.jsonl

1// Simple cost calculator (using per-1M-token unit prices)
2function estimateCostUsd(
3  promptTokens: number,
4  completionTokens: number,
5  inputPricePerM: number,
6  outputPricePerM: number,
7): number {
8  const inputCost = (promptTokens / 1_000_000) * inputPricePerM;
9  const outputCost = (completionTokens / 1_000_000) * outputPricePerM;
10  return Number((inputCost + outputCost).toFixed(6));
11}
12 
13// GPT-4o example
14console.log(estimateCostUsd(1200, 340, 2.5, 10.0)); // 0.0064

7 Ways to Cut Your Token Budget

• Keep the system prompt short: A system prompt resent on every request is the biggest hidden cost line at daily volume — a 500-token system prompt across 10,000 requests a day alone amounts to 5 million input tokens; without caching, all of it is billed at the standard price.
• Take advantage of caching: As of 2025-01-21, OpenAI's cached-input price was a fraction of standard input (for GPT-4o, $1.25/1M, half of the standard $2.50/1M) — use provider features that cache large, frequently repeated context (e.g. a fixed system prompt). Caching only works on the portion that stays byte-identical from the start of the request onward; if you break the ordering, the hit rate drops.
• Batch with the Batch API: For work that doesn't need a real-time response, OpenAI's Batch API runs at roughly half the standard price (in exchange for a 24-hour SLA) — overnight summarization, classification, or bulk content-generation jobs fit this model well.
• Summarize history instead of accumulating it: In long conversations, instead of resending the entire history with every message, apply periodic summarization; a history that grows with every new user message makes you pay the input cost of message 1 N times over by message N.
• Pick the right model: For simple tasks like classification or summarization, try cheaper models like GPT-4o mini or Claude 3.5 Haiku first and measure on your own task — a lower unit price by itself does not produce a lower total cost; in the calculation in the previous section, GPT-4o mini ran the same task at roughly one-seventeenth the cost of GPT-4o.
• Cap output length: Output tokens are almost always more expensive than input; setting an upper bound like max_tokens keeps the model from generating unnecessarily long responses. If you're using a reasoning model, remember this cap may not cover unseen reasoning tokens — check the provider's documentation.
• Track it with real usage data: Instead of relying on estimated averages, monitor daily/weekly cost with a log/metrics pipeline that aggregates the usage field returned from every request; surprise bills usually come from the gap between assumption and actual consumption.

FAQ

What is a token, and how many tokens does a word cost?

A token is the sub-word unit an LLM uses when processing text; it can be a whole word or just a piece of one. By OpenAI's rough rule of thumb, in English about 4 characters or 0.75 words correspond to one token — meaning 100 tokens roughly equals 75 English words (help.openai.com).

What happens when the context window fills up?

When the context window is exceeded, the request limit has been surpassed; OpenAI's support documentation recommends condensing the prompt or splitting the text into smaller chunks in this case (help.openai.com, 2025-01-19 archive). The model doesn't automatically "remember" old messages that don't fit in the window — that's your application layer's responsibility.

How is LLM cost calculated?

Cost is the sum of the input token count × the input unit price and the output token count × the output unit price. In the 2025-01-21 period, for example, GPT-4o's input was $2.50/1M and output was $10.00/1M (openai.com/api/pricing); for current figures, check the provider's live pricing page, since these values change over time.

Why does text in other languages cost more tokens than English?

One likely reason is that tokenizer vocabularies are trained predominantly on English text, so English sub-words appear more often as a single token in the vocabulary. As in OpenAI's Spanish example in its documentation ('Cómo estás' contains 5 tokens for 10 characters), the token-to-character ratio can come out higher in non-English languages (help.openai.com, 2025-01-19 archive) — providers don't publish a separate multiplier for non-English languages, so you need to measure it yourself with your own text via the provider's tokenizer tool.

How does cache pricing work?

Providers bill input that starts with the same prefix as a previous request as "cached input," at a lower unit price. On 2025-01-21, OpenAI's GPT-4o cached-input price was $1.25/1M, half of the standard input price ($2.50/1M) (openai.com/api/pricing). Caching applies only to the portion of the request that stays unchanged from the beginning.

When does the Batch API make sense?

It makes sense for work that doesn't require a real-time response and can be processed in bulk (e.g. overnight summarization/classification jobs). On 2025-01-21, OpenAI's Batch API ran at roughly half the standard price, with a 24-hour SLA (openai.com/api/pricing).

Update (September 2026)

This article's body is based on model and pricing data from 2025-01-21. Since that date, there have been fundamental changes to context-window and pricing architecture — per providers' official documentation (platform.openai.com/docs, platform.claude.com/docs, accessed 2026-09):

• Context windows grew: from the 128K-200K range in early 2025, by 2026 OpenAI's GPT-6 Astra reached a ~1,050,000-token context window, and Anthropic's Claude Fable 5.1 / Opus 5.5 / Sonnet 5 also reached 1 million tokens (Claude Haiku 4.5 stayed at 200K).
• Pricing is no longer a single tier: per GPT-6 Astra's documentation, prompts above 272,000 tokens bill the ENTIRE request at 2× input/cache rates and 1.5× output — the "flat unit price" logic of early 2025 no longer holds for flagship models.
• Model families and price tiers were refreshed: OpenAI's GPT-6 Astra/Sol/Luna (roughly 10-50 / 2-10 / 0.10-0.50 USD/1M); Anthropic's Claude Fable 5.1/Opus 5.5/Sonnet 5/Haiku 4.5 (roughly 10-50 / 4-20 / 2-10 / 1-5 USD/1M) are now in play.
• Cache economics changed: per Anthropic's pricing docs, Claude Fable 5.1's cache-read cost dropped to a quarter of Fable 5's; OpenAI split input/cache-read/cache-write into three price line items and added Batch (50% off) and Fast mode (2× surcharge) tiers.
• A new cost variable — reasoning effort: OpenAI's reasoning.effort parameter (none→max) directly affects token consumption and cost; Anthropic now exposes a similar effort parameter alongside adaptive/extended thinking (Fable 5.1 defaults to high, Opus 5.5 to medium, unsupported on Haiku 4.5).

The common outcome: the core math above (input × unit price + output × unit price) still holds, but "which unit price" can now depend on prompt length and the effort level you choose — checking the provider's live pricing page has become even more critical than in 2025.

Conclusion

Tokens, the context window, and cost math are the basic budgeting unit of any LLM integration — you can't estimate production cost without understanding these three. Keeping fixed cost items (system prompt, recurring context) small, taking advantage of caching, and choosing the right model/effort level can change the bill by orders of magnitude.

To dig deeper, you can check out related articles: to compare model choice against benchmark data, LLM Benchmarks 2026: MMLU, HumanEval, SWE-bench and Real-World Performance; to learn caching strategy in depth, Claude Prompt Caching: A 10x Cost-Reduction Guide; to understand when you need RAG instead of a large context window, RAG or Fine-Tuning? The Definitive Guide to Production LLM Decisions; to systematize prompt design, Prompt Engineering Patterns: Practical Techniques from a 10-Year Archive; and to see how the token budget compounds in tool-calling agent architectures, Agentic AI: Tool Use, Planner Loops, and Production Agent Architecture.

Sources

• OpenAI API Pricing (Wayback, 2025-01-21) — input/output/cached-input unit prices for GPT-4o, GPT-4o mini, o1, and the Batch API discount.
• Anthropic Models Overview (Wayback, 2025-01-19) — context window and pricing figures for Claude 3.5 Sonnet/Haiku and Claude 3 Opus.
• OpenAI Help Center — What are tokens and how to count them (Wayback, 2025-01-19) — token definition, the ~4 characters/0.75 words rule, an example of language-based tokenization differences (Spanish 'Cómo estás').
• OpenAI Help Center — Understanding and counting tokens (current) — token definition and the ~4 characters/0.75 words rough estimation rule.
• OpenAI Platform Docs — Pricing (current) — the 2026 model family and current unit prices.
• OpenAI Platform Docs — GPT-6 Astra model page — the 2×/1.5× pricing rule for prompts above 272K tokens, ~1,050,000-token context window.
• Anthropic Docs — Claude Models Overview (current) — context window and price band for Fable 5.1/Opus 5.5/Sonnet 5/Haiku 4.5.
• Anthropic Docs — Pricing — Claude Fable 5.1's cache-read price ($0.25/MTok) dropping to a quarter of Fable 5's ($1/MTok).