Models do not see letters or words. Before anything happens, your text is split into tokens: short chunks drawn from a fixed vocabulary. A common word like the is one token; a longer or rarer word like tokenization might be split into several. As a rough rule for English, one token is about four characters, and 100 tokens is roughly 75 words. Code, punctuation, and non-English text tokenize differently, often less efficiently.
Why you should care about a low-level detail
Tokens are not just an implementation detail. They are the unit almost everything is measured in:
- The [context window](/ai-coding-dictionary/context-window) is counted in tokens. A "200K context" means 200,000 tokens, not words or lines.
- Pricing is per token. You pay for tokens in and tokens out, so a verbose prompt or a giant pasted file has a direct cost.
- Limits bite in tokens. When an agent says it is running low on room, it is running low on tokens.
A practical consequence
Because tokenization is uneven, "small" inputs can be surprisingly expensive. A minified bundle, a wall of JSON, or a base64 blob can burn far more tokens than its character count suggests, while crowding out the code you actually want the model to focus on. Being deliberate about what you hand a model, and in what form, is really an exercise in spending tokens well.
Related terms
Context window
The context window is the maximum amount of text, measured in tokens, that a model can consider for a single request. It is a hard ceiling, and it is the main resource you manage when working with an agent.
Read definition →Model
A model is the trained artifact at the centre of every AI coding tool: a large file of numbers (parameters) that, given some text, produces the most likely continuation. When people say "which model are you using," this is the thing they mean.
Read definition →Inference
Inference is the act of running a trained model to get an answer: text goes in, a prediction comes out. Every message you send to a coding agent is an inference. It is the opposite end of the lifecycle from training.
Read definition →