What a token is and how a model counts text and code

A token is a chunk of text representation that a language model operates on. It is not the same as a word, a character, or a line of code. Common words and fragments often fit into a single token; rare names, long identifiers, JSON, and stack traces get split into several. Indentation, brackets, and punctuation count too. Different model families use different tokenizers: the same TypeScript snippet can produce a different number of tokens in two APIs.

The rough estimate "four characters ≈ one token" sometimes works on plain English prose and almost always lies on code, logs, and mixed-language text. Before sending an important request, count tokens with the tokenizer of the chosen family or with the counter in the API documentation. A few small teaching contrasts: a short comment in plain English; a TypeScript function with long names; compact JSON; a two-screen stack trace. The last one is almost always "heavier" than it looks to the eye.

Which tokens you pay for

Input tokens are everything the model receives: system instructions, your request, conversation history, attached files and repository fragments, and tool outputs inserted back into the context. Output tokens are the visible answer: a plan, an explanation, code, JSON. Some models also meter reasoning tokens separately: internal work may be billed differently from visible text, so "all output = what you read" is no longer true. Cached input is a repeated prefix that the API serves at a reduced price when the provider's conditions are met; it does not mean "the entire context is free."

Usage field names, units, and rules differ between OpenAI, Anthropic, Google, and others. Before budgeting, check the official pricing page as of the date of your calculation. Enterprise per-project and per-gateway accounting is covered in FinOps for LLM Gateway and in token budgets and chargeback.

The request cost formula

The general scheme:

cost ≈ input × input_price + output × output_price + cached_input × cached_price + other operations

If the rate is quoted per million tokens, convert your counts to millions. Input and output almost always cost different amounts. The total for a task includes not one call but a series: clarifications, a retry after red tests, a second agent pass.

Teaching example (prices are fictional, not a 2026 rate card). Suppose input = $0.0005/1K, output = $0.0015/1K, cached input = $0.0001/1K. One call: 40,000 input + 10,000 cached + 3,000 output → 40×0.0005 + 10×0.0001 + 3×0.0015 = $0.0255. Five such fix cycles without any growth in cache is already about $0.13 for the model alone, not counting human time.

Add the human layer. If each cycle takes ten minutes of review, five cycles are almost an hour. At an internal team rate even a modest hour easily outweighs the API bill. So "saving two sentences in the prompt" is almost never the main lever: cutting useless loops and noise in the input matters more. Plug in numbers from your billing and time tracking; the exercise is about multipliers.

Why context is the main hidden cost

The model decides based on what it was given. In AI development that includes the editor's system instructions, repository rules, history, open files, code search results, build errors, and tool outputs. As a task gets more complex, the context grows; some setups resend a large prefix on every step.

Separate four concepts. The context window is the upper limit the model can process in one call. Tokens actually sent are what went into this request. Context cost is how much of that was billed (including cache). Context quality is how much the data actually helped solve the task. A large window is a possibility, not an obligation to feed it a monolith. An extra file or a stale log raises the bill and sometimes makes the answer worse. How IDEs and agents assemble context under the hood is covered in the breakdown of AI IDEs.

Where tokens go in Cursor and similar editors

A typical loop (tools differ in the details; don't copy this as a Cursor specification): you state a task → the editor adds the system prompt and rules → picks files or fragments → the model plans → the agent reads more files or calls tools → results return into the context → code is generated or edited → tests or the linter run → errors go back to the model → the loop repeats.

A short user message can unfold into a large total context. An agent often makes several model calls per single "button press." A huge open file and a long history make each step more expensive. A subscription quota, internal credits, and direct API are different payment worlds; don't mix them in one table without footnotes.

A practical observation habit: for one task, note how many times the "thinking" model was called, which files clearly entered the context, and whether test runs fed logs back in. Even without full IDE telemetry, a week of such a diary shows where the team burns budget — exploring the repo, endless style fixes, or large diffs without acceptance criteria. More on agent loops in agentic engineering.

Why the same "feature" costs different amounts

The wider the context, the more calls, and the weaker the acceptance criteria, the heavier the cost tail. Manual rework after "almost done" is also part of the economics, even if API billing doesn't see it.

Prompt caching: when repeated context gets cheaper

Where the API supports prefix caching, a repeated stable chunk (system prompt, rules, a long specification) can be billed as cached input. A stable prefix is more useful than a constantly changing beginning: dynamic content in the first tokens often breaks cache hits. Caching does not replace trimming junk: you still pay (less) and still spend the window. Minimum size, TTL, and structure depend on the provider; check the documentation before building a budget on the cache.

Teaching comparison (again, fictional prices). The same logical context of 50,000 tokens: without cache, as ordinary input; with cache, 40,000 cached + 10,000 "fresh" input. When the cached price is several times lower than ordinary input, the difference over a series of identical calls becomes noticeable. Take the numbers from your own billing.

The window, history, and "memory" management

The context window is the limit of a single call. A product's long-term memory (notes, rules, a vector index) lives separately and enters the input only when a tool mixes it in. Working strategies: keep history current; summarize completed stages; store rules and architecture notes outside the chat; pass contracts and interfaces instead of a whole module; search for fragments instead of reading the whole repository; split the task into stages; throw out stale logs.

The trade-off is real: overly aggressive trimming causes missed requirements and new cycles, and the total bill grows. The goal is the minimum total cost up to an accepted result, not the shortest prompt in the log.

Why the cheapest model doesn't always save money

Two strategies. A cheap model with five rounds of edits and an hour of your review. A more expensive model that more often meets the criteria on the first or second pass. Compare: cost per call; cost per task; success rate; number of retries; human time; the share of code accepted without rewriting. The main metric is the cost of a successful result. The winner depends on the class of tasks, the quality of instructions, and acceptance; don't declare one model "best for everything." The shift in the engineer's role alongside models is covered in the programmer in the AI era.

Subscription or API: how to compare fairly

A subscription with a quota, in-product credits, pay-as-you-go API, separate limits per model and mode, enterprise plans: these are different contracts. Compare the monthly actuals, the number of people, intensity of use, access to the models you need, limit reset rules, surrounding tooling, the ability to allocate costs to projects, and the price of overage. You can't automatically say: "the subscription limit = N million API tokens worth X." Vendors count quotas in their own ways. Local models remove the cloud bill but move the cost to hardware and operations; see local MoE and hardware.

A practical experiment: one task, several models

Don't fill in the table with made-up winners. Use it as a protocol.

Sample task: add filtering and sorting to a React component of an existing ERP module without breaking types, API contracts, and tests.

1. The same starting revision of the project.
2. The same completion criteria.
3. Model, version, settings, and tool are fixed.
4. Available usage and cost are collected.
5. Calls and fix cycles are counted.
6. The same tests, linter, and build.
7. Independent quality assessment.
8. Repeat if results are unstable.
9. Separately: model money, wall-clock time, manual rework.

Without such a protocol, "Claude is cheaper than GPT for us" is an anecdote, not a measurement.

Agent economics: the price of a loop

An agent repeatedly calls the model and tools: search, reading, edits, tests, log analysis, more edits. The estimation model:

task cost = sum of the cost of all model calls + other billable operations

Spend limits: an iteration cap; explicit completion criteria; a per-task budget; a ceiling on log size; only relevant tests; a stop on a looping error; a mandatory change report. Code execution and external services may cost separately from tokens.

Budget-by-loop thinking. Research: three search and read calls, moderate input. Implementation: two calls that generate a patch. Verification: one call with a trimmed failing-test log. Six model steps total instead of "the agent spun for forty minutes until it hit a limit." If on the third loop the same linter error returns, it is cheaper to stop the agent and fix a project rule than to burn ten more attempts. The engineering of such loops is again agentic practice.

How to cut spend without losing quality

Request: the result and completion criteria; constraints and known interfaces; no copy-pasting requirements three times; a minimal reproducible example. Project: current rules; a short description of the architecture and verification commands; noise excluded from context; large changes in stages. Agent: a cycle ceiling; no giant logs "just in case"; search instead of reading everything; stop on criteria; research separate from implementation when that reduces retries. Model: a light one for formal chores; a strong one for architecture and tangled analysis; your own measurements; account for the cache in the rate.

The principle: don't save tokens at the price of new bugs. Trimming that breeds cycles costs more than generous but accurate context.

Team budget

Template parameters: number of developers; tasks per day; average number of calls per task; average input/output; cached share; rates; headroom for retries; other tools. A rough estimate:

month ≈ number of tasks × average task cost

More precisely, it is the sum of actuals from usage and models plus a reserve for heavy agent tails. Look not only at the average: the median and p95 matter, because a few expensive tasks eat the budget. Allocating costs across teams is again covered in the materials on token chargeback.

Common estimation mistakes

Counting tokens as words or lines; counting only the visible answer; forgetting retries; confusing the window with what was actually sent; believing that "the whole repository always gets sent"; treating the cache as free; comparing models only by input price; equating a subscription with an API; judging by a single lucky run; ignoring tests and manual rework; using outdated price lists; confusing the token bill with the full cost of development.

From tokens to the economics of AI engineering

Tools increasingly manage context on their own. The gains come from routing tasks between models, caching and reusing artifacts, and cost observability at the task and project level. The evaluation shifts from "price per call" to the cost of an accepted change and the time saved. No promises that "everything will be free in a year": count what you can measure today.

FAQ

Further reading

• LLM Gateway and FinOps
• Token budgets and chargeback
• Agentic engineering
• How AI IDEs work with code
• Local MoE and hardware
• The programmer in the AI era

Conclusion

Tokens measure model usage. Value is measured by an accepted, verified change. Context and cycles are often more expensive than a short question. Take rates from official pages as of the date of calculation; the teaching numbers in this article serve only to show the structure of the formula. Compare models on identical tasks. The main metric is the cost of correctly completed work, including verification and rework. Economical AI development is the engineering of context, tools, model choice, and acceptance, not a sport of shortening prompts.

This week one step is enough: take one real task from the backlog, write down the number of calls and the available usage, and calculate the cost up to green tests, without averaging "by eye."