Context engineering

It is the discipline of deciding what a model sees. A model can only work from the text in front of it, so the quality of any answer is capped by the context you assemble: the files, examples, history, and tool results you fit into a finite window. Getting that right is most of the work in any serious LLM system.

No. Prompt engineering is about wording a single instruction well. Context engineering is the larger job of choosing what information reaches the model at all: what to retrieve, what to leave out, how to compress history, and how to keep the window useful as a task grows. The prompt is one small part of the context.

The context window is the maximum amount of text, measured in tokens, that a model can consider for a single request. It is a hard ceiling and a shared budget: the system prompt, the history, the files, and the tool results all compete for the same space, so every token you add crowds out another.

RAG, retrieval-augmented generation, means pulling the material relevant to a request into the context so the model answers from it rather than guessing from memory. You need some form of retrieval whenever the answer depends on information the model was not trained on, like your own docs or code.

No. Embeddings and vector search are one option, good for fuzzy matching across a large corpus. But you can also let a model navigate a document tree, follow links in a knowledge graph, or read a file you point it at directly. The right retrieval is whatever puts the correct material in the window most reliably.

Chunking is splitting a long document into smaller pieces before you embed and retrieve them. The size and overlap of the chunks decide what can be found as a single unit, so it quietly makes or breaks a retrieval system. Chunk too big and you waste the window, too small and you lose the meaning.

The window is what the model can see right now, and it resets every session. Memory is an external store the system keeps and reloads into the window when it is relevant. The model itself remembers nothing between requests, so anything that has to persist lives in memory, not in the model.

Two reasons. Context accumulates like sediment, so old and irrelevant text crowds out what matters, which is context rot. And models attend best to the start and end of a long window while missing the middle. Compaction, clearing, and careful budgeting are how you fight both.

It is the tendency of a model to use information at the start and end of a long context well while missing what sits in the middle. It means a bigger window does not automatically mean better recall. If a fact matters, put it where the model actually looks rather than burying it halfway down.

Compaction is condensing older conversation history into a summary to reclaim window space while keeping the important gist. It is lossy by design, so the art is in summarising the trajectory and the decisions, not just the last few messages, so the agent can keep going without relearning everything.

Progressive disclosure is revealing detail to the model only when it is needed, through pointers and on-demand loading, instead of dumping everything into the window up front. You show what exists and its cost, then load the specific piece when the task calls for it. It saves both space and attention.

Treat the window as a budget. Load information only when a task needs it, point to large files instead of pasting them, compact or summarise old history, and hand off to a fresh session with a short state document when the window fills. The goal is to keep the highest-value tokens in and everything else out.

Chunking, almost always. Most retrieval problems start at the cut: pieces too small to be meaningful, too large to be precise, or severed from the context that made them findable. Fix that first, then add contextual retrieval so each chunk carries where it came from. Only after that is hybrid search or reranking worth reaching for. Swapping vector databases is almost never the answer.

Usually one bad passage in the context rather than anything wrong with the model. Context pollution is a single wrong or outdated item steering everything downstream of it, and the model reasons from it correctly, which is what makes the output so plausible. When something looks wrong, read the context that was actually assembled before touching the prompt. The false premise is usually sitting there in plain text.

That is goodharting, and it arrives faster in LLM systems than people expect because a prompt can be reshaped in seconds against a fixed score. The defences are holding out cases you never tune against, keeping metrics that trade off against each other so you cannot inflate both, and reading actual outputs by hand. A rising score with flat user-reported quality is the diagnostic, and you should believe the users.

You measure it. Build a set of real cases with known good answers and score your system against them, using a model as a judge for the fuzzy ones. Evaluation turns context engineering from guesswork into something you improve on purpose, one change at a time.

A workflow follows a path you designed in advance; an agent decides its own path at run time by calling tools in a loop. Use a workflow when the steps are known and you want reliability, and an agent when the path genuinely depends on what it finds along the way. Choosing correctly is the first context-engineering decision.

No. Any time you work with a model, from a coding agent to a data pipeline, you are deciding what it sees. Retrieval is one part. Memory, history management, tool results, specs, and handoffs are all context engineering. It is the substrate under every serious LLM application, not a single feature.