Two teams build the same AI feature with the same model. One is reliable and sharp; the other is vague and unpredictable. The difference is almost never the model — it is what information they put in front of it. That craft is context engineering, and it is quietly the most important skill in building AI systems that actually work.
- Understand why the model's output is only ever as good as its input context
- Treat the context window as a limited budget and spend it well
- Select, order, and trim information so the model focuses on what matters
- Decide what an AI system should remember and what it should forget
The model only sees its context
An LLM has no memory of you, no access to your files, no awareness of anything except the text you place in its context window right now. Whatever is in that window, it uses. Whatever is missing, it cannot use — and will often guess to fill the gap.
This leads to the golden rule: garbage in, garbage out; gold in, gold out. If the model gives a vague answer, the usual cause is not a weak model — it is that the right information was never put in front of it.
- The model can only use what is in its context window at that moment
- Missing information leads to guessing; vague output usually means poor context, not a weak model
- The key question is "what does it need to see?" not "how do I phrase it?"
The context window is a budget
The window has a fixed size (a token limit). Everything competes for that space: your instructions, the retrieved documents, the conversation so far, examples, tool results. You cannot pour in everything — and even if you could, you should not.
Why not? Two reasons. First, cost and speed — more tokens means slower, pricier calls. Second, and more surprising: too much context hurts quality. Bury the one relevant paragraph inside fifty irrelevant ones and the model's attention gets diluted; it can miss the very fact you included. Models even tend to overlook things stuck in the middle of a long context.
STUFFED WINDOW FOCUSED WINDOW
[50 pages of docs] [the 2 relevant paragraphs]
[whole chat history] [the last 3 turns that matter]
[every tool result] [the one result needed now]
→ slow, costly, model → fast, cheap, model
misses the key line sees exactly what matters
- The context window is a fixed budget every piece of information competes for
- Too much context is slow, expensive, and actually lowers accuracy
- The goal is the right information, focused — not the most information
- Why can adding more documents to the context make the answer worse?
- Rewrite the question "how do I phrase this prompt?" the way a context engineer would ask it.
Select, order, and compress
Good context engineering is three moves, done well:
Select — include only what's relevant
Pull in the documents, history, and tool results that actually bear on this task, and leave the rest out. This is where RAG (last course) earns its keep — fetching just the relevant passages instead of the whole library.
Order — put the important things where they're seen
Models pay most attention to the start and end of the context, less to the middle. Place the key instruction and the most relevant facts prominently — not buried in the middle of a wall of text.
Compress — shrink without losing meaning
Summarise long histories, trim boilerplate, replace ten verbose paragraphs with the three sentences that matter. Compression frees budget for what counts.
- Select: include only what's relevant to this task (RAG helps here)
- Order: put key instructions and facts where the model attends most (start/end)
- Compress: summarise and trim so the budget holds signal, not filler
Memory — what to keep, what to forget
Conversations and tasks pile up information faster than the window can hold. So a system must decide, continuously, what to carry forward and what to let go. This is memory design, and it is pure context engineering over time.
A simple, powerful pattern: keep a running summary of what matters (the user's goal, key decisions, important facts) and drop the raw details once they are summarised. The model always sees a compact, up-to-date picture instead of an ever-growing transcript.
This is also the bridge to the next course. Once you are deciding what the model sees and remembers, you are really designing the system around the model — its tools, memory, and rules. That system is the harness, and it is next.
- Long tasks overflow the window, so a system must choose what to carry forward
- A running summary keeps the essentials compact and current
- Good memory means keeping what will matter later in the smallest faithful form
- Name the three moves of context engineering and what each does.
- Why is "remember everything" a bad memory strategy?