# What does a token cost? Context length, input, and output

[Skip to content](#lm-inhoud)Network/[NL](/en/wat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer)EN[Hubhub.llmnet.nlCompare models on task, language, cost and licence.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organisation, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer&text=What%20does%20a%20token%20cost%3F%20Context%20length%2C%20input%2C%20and%20output)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer)[](https://www.reddit.com/submit?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer&title=What%20does%20a%20token%20cost%3F%20Context%20length%2C%20input%2C%20and%20output)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer&text=What%20does%20a%20token%20cost%3F%20Context%20length%2C%20input%2C%20and%20output)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer)[](https://www.reddit.com/submit?url=https%3A%2F%2Fleren.llmnet.nl%2Fen%2Fwat-kost-een-token-rekenen-aan-contextlengte-invoer-en-uitvoer&title=What%20does%20a%20token%20cost%3F%20Context%20length%2C%20input%2C%20and%20output)[](#)By Ivo Donker — compiled with AI assistance (Claude & Gemini) · Last updated: August 7, 2026

 
 
# What does a token cost? Calculating context length, input, and output

 By Ivo Donker — compiled with AI assistance (Claude & Gemini) · Last updated: August 7, 2026

Prerequisites: This article builds on fundamental knowledge of language models within the educational curriculum. Consult [Tokenization Explained](https://leren.llmnet.nl/en/tokenisatie-uitgelegd) to understand precisely how text is broken down into computable subword units. This article explains which algorithms split texts, which is essential for estimating token counts. See [Parameters and Weights](https://leren.llmnet.nl/en/parameters-en-gewichten) for insight into the internal computational structure of large language models. This page provides background information on how model size and memory usage are related. Open the [AI Glossary](https://leren.llmnet.nl/en/ai-begrippenlijst) if specific technical terms require clarification. This overview helps readers quickly look up definitions of industry jargon.

Article reviewed on 2026-08-07.

## 1. Introduction: The necessity of a clear token economy

In the proposed learning path for developers, this guide serves as module 1 (Foundations, orientation). Anyone who starts building applications on top of Large Language Models (LLMs) via API interfaces quickly realizes that the financial structure differs fundamentally from traditional software. In classic cloud architectures, costs are typically calculated per CPU hour, per gigabyte of memory usage, or per outbound network transaction. With language models, by contrast, the token constitutes the primary billing unit. Although the concept of a token is frequently touched upon in introductory documentation, the exact financial ramifications for production environments often remain underexplored.

The question "what does a token cost?" cannot be answered with a single fixed number. The cost of an API call depends on the ratio between the submitted text and the generated response, the total size of the context, and the potential use of reuse mechanisms such as caching. Deploying an application to production without a calculation model can lead to monthly invoices that are orders of magnitude higher than initial estimates. To avoid surprises, this article dissects the token economy down to the euro cent using mathematical formulas and practical calculation examples.

The objective of this module is to provide a mechanical understanding of the cost structure. The article refrains from recommending for or against the use of long context windows or specific commercial providers. Instead, it offers the computational rationale and objective decision rules needed to configure engineering workflows with predictable budgeting.

## 2. What makes up the bill: Input versus output tokens

When a request is sent to an LLM API, the provider splits billing into two main categories: input tokens and output tokens. The total cost of a single API call is defined by the sum of both variables, multiplied by their respective rates.

Input tokens include all information that the client sends to the model. This consists of the system prompt (instructions for the model's role and behavior), contextual information (such as retrieved documents, database results, or code files), and the complete previous conversation history. Output tokens are the tokens that the model generates during the inference phase and returns to the client.

Read [Per-token pricing models explained on hub.llmnet.nl](https://hub.llmnet.nl/en/prijsmodellen-per-token-uitgelegd) for a detailed overview of commercial contract types with API providers. That page compares pay-as-you-go rates with reserved capacity and volume discounts.

With almost every commercial API provider, the rate for output tokens is significantly higher than that for input tokens. An average ratio is that output tokens are three to four times more expensive per unit. This price difference is not arbitrary, but stems directly from the hardware processing on the provider's graphics processing units (GPUs):

- Prompt Prefill Phase (Input): The input tokens are processed by the model in one large parallel matrix multiplication. Because all input tokens are known simultaneously, the compute power of the GPU (the Tensor Cores) can be optimally utilized. This phase is primarily compute-bound and runs very efficiently.

- Autoregressive Generation Phase (Output): The model must generate output tokens one by one. Each newly generated token serves as input for the next step. As a result, for each individual token, the model must reload all billions of parameters from the GPU's VRAM into the compute cores. This phase is strictly memory-bandwidth bound, making hardware utilization per token far less efficient.

See [Context engineering explained](https://leren.llmnet.nl/en/context-engineering-uitgelegd) for techniques to efficiently structure the contents of the context window. On that page, you will learn how to remove redundant formatting and history without losing information.

## 3. The impact of context length on pricing trends

A common misconception when building LLM applications is the idea that you only pay for the 'new' text typed into a chat conversation. However, the API interfaces of modern language models are stateless. The model does not remember previous interactions between consecutive HTTP requests.

This means that with every interaction in an ongoing conversation, the full history must be included again. In a chat application where the user asks ten consecutive questions, on the tenth question the client sends not only that tenth question, but also the system prompt, questions 1 through 9, and answers 1 through 9. The number of input tokens therefore grows cumulatively with each interaction.

In addition, the scalability of the attention matrix within the transformer architecture plays a role. The computational complexity of the standard self-attention operation scales quadratically with the length of the context. While modern variants such as FlashAttention optimize this compute at the hardware level, the memory footprint of the key-value pairs (the KV cache) continues to scale linearly with every additional token in the context. When the context window expands from 2,000 to 100,000 tokens, the amount of data that must remain pinned in GPU memory per request increases by a factor of fifty.

When a developer loads a 100,000-token document into the context window to ask five sequential questions about it, the developer pays the input costs for those 100,000 tokens not once, but five times. Without additional optimizations, this leads to a steep, cumulative cost increase.

## 4. Three Worked Examples with Specific Euro Amounts

To make the financial mechanics transparent, three worked scenarios are provided below. Important Disclaimer: All mentioned euro amounts and rates in these examples are hypothetical assumptions within a calculation model as of reference date 2026-08-07. These amounts serve solely to illustrate the calculation methodology and do not represent actual market prices or guaranteed quotations.

### Assumptions for the Calculation Models (Reference Date 2026-08-07)

For the calculations below, we use the following hypothetical base rates for a hypothetical advanced language model:

- Uncached input tokens: €0.0025 per 1,000 tokens (€2.50 per 1,000,000 tokens).

- Cached input tokens: €0.0005 per 1,000 tokens (€0.50 per 1,000,000 tokens).

- Output tokens: €0.0100 per 1,000 tokens (€10.00 per 1,000,000 tokens).

### Scenario A: A Short Chat Call (Lightweight Interaction)

In this scenario, a user asks a simple question to a customer support bot. The context consists of a brief system prompt and the user query. No caching takes place.

- Input: 1,800 tokens (system prompt) + 200 tokens (question) = 2,000 input tokens.

- Output: 200 tokens (response).

- Input cost: (2,000 / 1,000) * €0.0025 = €0.0050.

- Output cost: (200 / 1,000) * €0.0100 = €0.0020.

- Total per call: €0.0070.

### Scenario B: Dutch Case Study – RAG on a Legal Knowledge Base (30k Context)

A Dutch municipality uses a Retrieval-Augmented Generation (RAG) system to search policy documents. For every query submitted by a civil servant, the system retrieves 30,000 tokens of relevant Dutch laws and regulations and appends them to the prompt. The model generates a 500-token summary response.

Variant B1 (Uncached): Each request contains unique policy documents, or the system does not utilize context caching.

- Input: 30,000 tokens.

- Output: 500 tokens.

- Input cost: (30,000 / 1,000) * €0.0025 = €0.0750.

- Output cost: (500 / 1,000) * €0.0100 = €0.0050.

- Total per call (uncached): €0.0800.

Variant B2 (Cached): The 30,000 tokens of policy text form a fixed, static baseline that is repeatedly queried by multiple civil servants throughout the day. The provider detects the static segment and applies the lower cache pricing tier.

- Cached input: 29.000 tokens at €0,0005 per 1k = €0,0145.

- New input (uncached): 1.000 tokens at €0,0025 per 1k = €0,0025.

- Output: 500 tokens at €0,0100 per 1k = €0,0050.

- Total per call (cached): €0,0220. (A 72.5% reduction in total call costs).

### Scenario C: Document processing (100k context)

A financial analyst inputs a complete annual report of 100.000 tokens and asks the model to generate a structured risk analysis of 2.000 tokens. This is a one-off, heavy processing call without caching.

- Input: 100.000 tokens.

- Output: 2.000 tokens.

- Input costs: (100.000 / 1.000) * €0,0025 = €0,2500.

- Output costs: (2.000 / 1.000) * €0,0100 = €0,0200.

- Total per call: €0,2700.

### Comparative overview of calculation examples (Assumptions as of 2026-08-07)

The table below summarizes the calculation results and shows the financial impact at a scale of 1.000 requests per day.

Scenario | 
Context length (Input) | 
Input costs | 
Output costs | 
Cached? | 
Total per call | 
Costs per 1.000 calls/day | 

A. Short chat | 
2.000 tokens | 
€0,0050 | 
€0,0020 | 
No | 
€0,0070 | 
€7,00 | 

B1. RAG Policy (Uncached) | 
30.000 tokens | 
€0,00750 | 
€0,0050 | 
No | 
€0,0800 | 
€80,00 | 

B2. RAG Policy (Cached) | 
30.000 tokens | 
€0,0170 | 
€0,0050 | 
Yes (29k tokens) | 
€0,0220 | 
€22,00 | 

C. Document processing | 
100.000 tokens | 
€0,2500 | 
€0,0200 | 
No | 
€0,2700 | 
€270,00 | 

## 5. When caching works (and what the limitations are)

To reduce the processing costs of repeated large contexts, various API providers offer context caching mechanisms. The technical foundation of this is reusing the already computed KV cache on the provider's GPU servers, so that the model does not have to re-execute the prompt prefill phase for known text segments.

Study [KV cache architecture](https://leren.llmnet.nl/en/kv-caching-opbouw) for the deeper technical inner workings of the Key-Value cache in transformer architecture. This article explains how activation values are retained in GPU memory during inference.

Consult [Context caching explained on hub.llmnet.nl](https://hub.llmnet.nl/en/context-caching-uitgelegd) to evaluate which API providers support automatic or manual caching and what the financial thresholds are. On that platform, you will find market-wide comparisons of cache integrations.

### The mechanics of price reduction with caching

When a client submits a request where the first $N$ tokens are identical to a previously submitted request, the provider can read the KV cache directly from the memory address. This yields two benefits: latency (time to first token) drops drastically, and the provider charges a reduced rate for the cached input tokens (in the sample calculation, an 80% discount).

### Weaknesses and limitations of context caching

Context caching is not a universal solution and comes with clear operational drawbacks:

- Strict prefix dependency: With most providers, caching works exclusively from the very first token of the request. If you place a variable value (such as a unique user ID or a shifting timestamp) at the top of your prompt, the cache match is invalidated for the entire subsequent 100,000 tokens of text. Dynamic content must therefore always be appended to the end of the prompt.

- Minimum volume thresholds: Many API providers enforce a minimum threshold before caching is activated (for example, at least 1,024 or 32,768 tokens). For short prompts, caching therefore provides no financial benefit whatsoever.

- Lifespan and Cache Expiration (TTL): The stored KV cache is retained in GPU memory for only a limited time (Time-To-Live), often ranging from 5 minutes to several hours. If no new requests with the same prefix arrive during that period, the cache is purged. At low processing volumes, developers sometimes even pay additional 'storage fees' per hour to keep the cache alive, which for sporadic usage can end up being more expensive than uncached requests.

- No discount on output tokens: Caching applies strictly to the input phase. Processing output tokens continues to be billed at the full, premium rate.

## 6. The hidden costs in production environments

Developers who budget based on ideal test scenarios frequently face unexpected expenses in production environments. These hidden costs arise from operational side effects that silently multiply the number of processed tokens.

### 1. Unpredictable output lengths

While a developer can control the input length fairly accurately, the length of the generated output is inherently stochastic. When a model is triggered into an infinite loop, generates elaborate JSON structures, or outputs extensive reasoning (chain-of-thought), the most expensive tokens (the output tokens) surge dramatically. Without strictly configuring the parameter max_tokens a single runaway request can consume a hundred times the budgeted output costs.

### 2. Agentic loops, tools, and automatic retries

In modern AI architectures, models are deployed as 'agents' that autonomously invoke functions and tools. When a model generates a malformed function call, the application does not simply halt. The error message is fed back into the context, after which the model makes another attempt. Behind the scenes, a single user prompt can consequently result in four or five sequential API calls. Including the context resent with each iteration, the input token volume multiplies exponentially.

### 3. Rate limits and the financial impact of retried requests

When a production application hits an API provider's limits (Rate Limits expressed in Requests Per Minute or Tokens Per Minute), requests fail with HTTP status code 429. If the client application is not properly configured with an 'exponential backoff' strategy, aborted or failed requests are immediately resent. For requests canceled midway through processing, some providers still bill for the input tokens already processed, leading to direct capital waste.

Go to [Rate limits and costs on api.llmnet.nl](https://api.llmnet.nl/en/rate-limits-en-kosten) to see how processing limits indirectly affect your operational expenses. This overview helps developers design fault-tolerant retry mechanisms.

Visit [Monitoring costs on api.llmnet.nl](https://api.llmnet.nl/en/kosten-monitoren) for practical guides on setting up budget notifications and automated circuit breakers. This resource demonstrates how to prevent unexpected cost overruns in production systems.

## 7. Measure instead of assume: Analyzing usage per call

To maintain control over token economics, developers should not rely on word-count-based estimates. The ratio between the number of words in a text and the number of generated tokens varies considerably across languages and input types. Dutch texts consume on average 15% to 40% more tokens than English texts for the exact same content, as tokenizer vocabularies are primarily trained on English-language corpora.

### Reading API metadata

Virtually every API provider includes a JSON payload with the exact request telemetry alongside a successful response. Rather than estimating token counts internally, a production system should read these values directly and store them in its own log files.

A standard response metadata object conceptually looks like this:

- prompt_tokens: The exact number of input tokens processed.

- completion_tokens: The exact number of output tokens generated.

- cached_tokens: (If applicable) The portion of prompt_tokens retrieved from the cache.

- total_tokens: The sum of input and output tokens.

Check out [Token usage normalization across providers on api.llmnet.nl](https://api.llmnet.nl/en/token-usage-normalisatie-providers) to understand how response metadata across different APIs can be transformed into a single standard format. That page includes examples for consolidating telemetry from diverse API responses.

### Price per token versus cost per task

A crucial step in cost analysis is shifting the metric from "price per token" to "cost per completed task". A cheaper model per token is not necessarily cheaper per task. If a smaller, cheaper model requires 1,500 output tokens and three retries to correctly deliver a structured JSON file, whereas an expensive, larger model performs the same task flawlessly in 200 output tokens, the more expensive model is net cheaper for overall business operations.

Consult [Cost per task on benchmark.llmnet.nl](https://benchmark.llmnet.nl/en/kosten-per-taak) to compare the actual costs of varying model sizes per specific business task. This benchmark measures how much money a completed task costs instead of individual tokens.

## 8. Running Locally versus API Usage: The Physical Compute Balance

When estimated project API costs increase, organizations frequently consider switching to locally hosting open-weight models (such as Llama or Mistral variants) on their own hardware or via dedicated cloud instances (for example, with vLLM or Ollama). Within the token economy, this means switching from a variable operational cost model (OpEx) to a fixed investment and infrastructure model (CapEx / fixed OpEx).

### Comparing cost structures

When using a commercial API, you pay exclusively for the tokens actually processed. If a system is not used overnight, the API costs are zero euros. When hosting a model locally or on a dedicated GPU server (such as an Nvidia H100 or A10G instance), you pay a fixed hourly rate, regardless of whether the GPU is under 100% load or completely idle.

The cost price per token for a local GPU setup can be calculated by dividing the server's total operational costs by the total token throughput over a given period:

- Hardware depreciation or server rental: The fixed monthly amount for the dedicated hardware.

- Power consumption: The actual power consumption of the GPUs, CPUs, and cooling systems under load, multiplied by the local kWh price.

- Management overhead: The labor costs of the developers or system administrators responsible for scaling, securing, and maintaining the inference cluster.

Read [Power consumption for local AI on gids.llmnet.nl](https://gids.llmnet.nl/en/stroomverbruik-lokale-ai) for the physical calculation of kWh consumption, hardware depreciation, and cooling for self-hosted servers. That guide provides the necessary parameters to weigh the TCO of local hardware against cloud APIs.

### When is local hosting financially advantageous?

A local setup is mathematically only advantageous when GPU utilization is continuously high (for example, with constant batch processing of millions of documents) and when context lengths fit within the available VRAM of the graphics cards. With fluctuating workloads or extremely large context windows, API providers remain more financially efficient due to their economies of scale and the dynamic sharing of GPU capacity across thousands of customers.

## 9. Decision rules and limitations of this article

### Synthesis: Decision rules for builders

To keep the token economy manageable within software architectures, the following quantitative decision rules can be applied:

- Separate static and dynamic context: Place fixed instructions, documents, and templates at the beginning of the prompt and dynamic variables at the end to make optimal use of automatic context caching.

- Explicitly limit output length: Always set a hard upper bound on the desired response tokens via the API parameters to prevent cost overruns.

- Select the model based on task complexity: Use lightweight, low-cost models for classification, extraction, and simple text processing. Reserve heavier, more expensive models for complex reasoning steps and final synthesis.

- Prune conversation history: Implement a sliding window or summarize previous conversation turns instead of passing the entire, unconstrained chat history with every new query.

- Monitor at the task level: Log the exact values from the API metadata and calculate costs per successfully completed business transaction rather than per individual API call.

### Weak points and limitations of this article

In accordance with editorial guidelines, we highlight the inherent limitations and weaknesses of the analyses in this article:

- Market price volatility: Pricing from AI providers changes frequently. The rates and calculation examples used in this article are explicitly based on hypothetical assumptions as of 2026-08-07. They cannot be used as a guarantee of current commercial rates.

- Language-specific discrepancies: The exact cost per page of text varies by language. Because tokenizers process English more efficiently than Dutch, all token count assumptions depend on the language used and the specific tokenizer of the selected model.

- Simplification of hardware architecture: In practice, providers apply additional optimizations (such as Quantization, Speculative Decoding, and PagedAttention) that affect the internal cost price. This article addresses API costs from the perspective of an external user and ignores the internal margin structures of suppliers.

- Exclusion of network egress and latency costs: The calculation models focus exclusively on token rates. Any costs related to network connectivity, cross-region data transfers, or operational latency impacts are not included in the formulas.

Article reviewed on 2026-08-07.

Up next: Now that you have mastered the basics of the token economy, you can dive deeper into the practical optimization of contexts and memory structures. Read [Context engineering explained](https://leren.llmnet.nl/en/context-engineering-uitgelegd) to learn how to structure prompts for minimal token overhead, or explore [KV-caching architecture](https://leren.llmnet.nl/en/kv-caching-opbouw) for an in-depth technical analysis of GPU-level memory optimization.