An embedding turns a piece of text into a list of numbers. That sounds unremarkable until you notice what the numbers do: words that mean similar things end up close together, and the directions between words turn out to carry meaning of their own.
The clusters are the easy part to see. Animals sit near other animals, countries near other countries. The more surprising property is that relationships become directions. The step from "man" to "woman" is roughly the same step as the one from "king" to "queen", so you can do arithmetic on meaning: take king, subtract man, add woman, and you land next to queen.
Try the analogy presets below, then switch to 3D and drag to orbit the cloud, which is how you tell that the structure is genuinely spatial rather than an artefact of how it was drawn. The nearest-neighbour readout under the plot uses cosine similarity, direction rather than distance, because embedding length mostly tracks word frequency and direction is where the meaning lives.
A necessary caveat: the vectors here are illustrative, hand-built to show the geometry clearly. Real embeddings have hundreds or thousands of dimensions, and no one can look at those directly. These were authored in three so the geometry survives being drawn, which is the compromise every picture of an embedding space makes.
The maths
- Cosine similarity
The measure behind the nearest-neighbour readout. It compares direction only, ignoring length, which is why embeddings are usually compared this way rather than by Euclidean distance.
- Analogy arithmetic
The parallelogram the presets draw. king minus man plus woman lands near queen because the man-to-woman offset and the king-to-queen offset are approximately the same vector.
- The 2D projection
The flat view projects onto the two directions of greatest variance, which is principal component analysis. Both axes then share one scale on screen, so equal steps in the space are drawn as equal steps in the picture.
Related terms
- TokenA token is the unit of text a model reads and writes: a chunk that is usually part of a word, not a whole word or a single character. Everything is measured in tokens, including your context window and your bill.
- InferenceInference is the act of running a trained model to get an answer: text goes in, a prediction comes out. Every message you send to a coding agent is an inference. It is the opposite end of the lifecycle from training.
- ModelA model is the trained artifact at the centre of every AI coding tool: a large file of numbers (parameters) that, given some text, produces the most likely continuation. When people say "which model are you using," this is the thing they mean.
- ParametersParameters are the learned numbers (weights) inside a model that hold everything it appears to know. The count of them is what people mean by model size, and they are fixed once training ends.