What is Context Engineering?

What it is: Context engineering is the practice of carefully designing and managing all the information fed to an AI model to get better, more reliable outputs.
Who it’s for: Anyone learning AI terminology
Best if: You’ve seen this term and want a clear explanation
Skip if: You already work with this concept daily

Context engineering vs prompt engineering

For the first wave of generative AI, the practical skill everyone talked about was prompt engineering — writing one clever instruction, sticking it into a chat box, and crossing your fingers. That works for one-shot tasks like “rewrite this email.” It falls apart the moment you try to build something real, like a support bot that has to remember your refund policy, look up a customer’s order, and stay polite for twenty turns of conversation.

Context engineering is the wider job. It is the practice of deciding everything the model sees before it produces an answer: the system prompt, the conversation so far, the examples, the documents you fed it, the tools it can call, and the user’s latest message. Shopify CEO Tobi Lütke put it on the map in 2024 when he said the real skill in AI is “the art of providing all the context for the task to be plausibly solvable by the LLM.” Andrej Karpathy echoed the framing soon after, and Anthropic now uses “context engineering” as a first-class concept in their developer documentation.

The shift in thinking is small but important. Prompt engineering asks, “how do I word this question?” Context engineering asks, “what is the entire information environment my model needs in order to answer well?” One is a sentence. The other is a system.

What goes into context

The “context” a model sees on any given turn is just text — but that text is assembled from several distinct pieces, and a context engineer’s job is to choose what belongs in each one. Anthropic and OpenAI both describe roughly the same set of inputs:

• System prompt. The standing instructions: who the assistant is, what it is allowed to do, what tone it should use, what it must never do. This is the “rules of the game” layer.
• Examples (few-shot). A few demonstrations of the task done correctly. For a classification bot, that might be five labelled emails. Examples often beat long explanations because the model can copy a pattern more reliably than it can follow abstract rules.
• Retrieved documents. Knowledge pulled in on the fly — a help-centre article, a product spec, a slice of a PDF — usually via retrieval-augmented generation (RAG). This is how you give the model facts it was never trained on, like your own pricing or yesterday’s meeting notes.
• Conversation history. Prior turns between the user and the assistant. Without history, every reply is amnesia.
• Tool definitions and tool results. The list of functions the model is allowed to call (search the database, send an email, look up an order) and the JSON that comes back when it does. Tool output is itself context — and often the largest, messiest part.
• The user’s current message. The actual question.

All of this has to fit inside the model’s context window — the fixed-size text buffer the model can read at once. Modern frontier models like Claude Sonnet 4.5 and GPT-5 advertise windows of 200,000 to 1 million tokens, which sounds enormous until you start stuffing a knowledge base into it. Every piece you add costs money, slows the response, and competes for the model’s attention.

The art of trimming context

The instinct of most beginners is to throw more in: more examples, more documents, more rules. Experienced builders do the opposite. Three forces push you toward less:

• Cost. APIs charge per token of input. A 50,000-token prompt sent ten thousand times a day adds up fast. Anthropic’s prompt caching feature can recover a lot of that by reusing the unchanging prefix of a prompt, but only if you structure context with caching in mind — stable parts first, volatile parts last.
• Latency. Bigger inputs take longer to process. A bot that needs three seconds to think is fine; one that needs thirty is dead on arrival.
• Quality. Models genuinely struggle when context is bloated. Researchers call this the “lost in the middle” effect: a relevant fact buried on page 40 of a 60-page dump is often ignored, even though the model technically read it. Less context, ruthlessly chosen, beats more context dumped wholesale.

Good context engineering looks more like editing than writing. You start with everything you think might help, then cut until the model still gets the right answer.

A real example

Imagine a customer-support bot for an online shoe store. A user types: “My order hasn’t arrived and it’s been three weeks.”

A naive setup would just forward that sentence to the model. A context-engineered setup builds something like this before the model sees the question:

• System prompt: “You are a support agent for SoleMate. Be warm, brief, and never promise refunds you cannot authorise. Escalate any dispute over $200 to a human.”
• Retrieved policy: the two paragraphs of the shipping policy that mention delays — not the whole 12-page document.
• Tool result: the model just called lookup_order(user_id) and got back {"order_id": 4421, "status": "in_transit", "carrier": "DHL", "shipped": "2026-04-08"}.
• Conversation history: the previous two turns, where the user already said they live in Spain.
• User message: the original complaint.

The model now has everything it needs to write one sensible reply: acknowledge the wait, confirm the order is in transit with DHL, note the typical delivery window for Spain, offer to open a trace, and stop short of promising a refund. None of that quality came from a clever prompt. It came from the context that was assembled around the prompt.

10 Context Engineering Patterns Most Practitioners Miss

• Hierarchy of context: instruction, persona, examples, task. Order matters. Instructions first, persona second, examples third, task last. Model attention follows the hierarchy.
• Negative examples explicit, not implicit. Showing what NOT to do beats hoping the model infers it from positive examples.
• Few-shot example diversity over count. 3 diverse examples beat 10 similar ones. Diversity teaches the pattern; redundancy wastes tokens.
• Output format pre-specification. Specifying output structure upfront produces dramatically better adherence than parsing after.
• Constraint placement matters. Hard constraints near the task; soft preferences earlier. Model attention to constraints follows recency.
• Token-budget allocation across sections. Long contexts need allocation discipline: how many tokens for instructions vs examples vs task. Most users overflow inadvertently.
• Context compression via summarization. Long sources compressed before inclusion. Summarization-then-grounding beats raw long-context for most tasks.
• RAG retrieval quality over quantity. 3 highly-relevant retrieved passages beat 10 mediocre ones. Retrieval ranking matters more than corpus size.
• Self-consistency checks via second pass. First-pass output checked against constraints in a second pass. Quality jumps for low marginal cost.
• Context versioning over time. Production context engineering versions evolve. Tracking which prompt versions produced which outcomes is necessary infrastructure.

Common mistakes

• Dumping the whole knowledge base. Retrieve the relevant sections, not the entire wiki. Quality goes down as token count goes up.
• Forgetting to update the system prompt when the product changes. If your refund window is now 60 days but the system prompt still says 30, the model will lie confidently.
• Putting volatile data at the top. Prompt caching only helps when the unchanging parts come first. Putting the user’s name in the system prompt invalidates the cache on every turn.
• Mixing conflicting instructions. Three different layers each saying “always be formal” and “match the user’s tone” produce mush. Decide once, in one place.
• Skipping evaluation. Context that “feels” tight on three test cases can fall apart on the fourth. Run evals after every meaningful context change.

Related terms

• What is Prompt Engineering?
• What is a Context Window?
• What is a System Prompt?
• What is RAG (Retrieval-Augmented Generation)?
• What is Few-Shot Learning?
• What Are Evals?

What is Context Engineering?

Context engineering is the discipline of designing, curating, and managing the information that gets passed to an AI model alongside your request. While prompt engineering focuses on how you phrase your question, context engineering goes further — it’s about what background information, examples, documents, and system instructions you include to shape the AI’s behavior and output quality.

Think of it this way: a prompt is what you ask, but context is everything the AI knows when it answers. If you hand someone a question with no background, you get a generic answer. If you hand them the same question along with relevant documents, examples of what good answers look like, and specific instructions about your situation, you get something far more useful. Context engineering is the art of doing that systematically.

This has become especially important as AI models get larger context windows — the amount of text they can process at once. With models now accepting hundreds of thousands of words, knowing what to include (and what to leave out) is a critical skill.

Why It Matters

Context engineering matters because the quality of AI output is directly tied to the quality of its input context. Teams that master context engineering get dramatically better results from the same models that underperform for others. It’s the difference between an AI that gives generic advice and one that gives expert-level, situation-specific answers.

As AI agents take on more complex tasks, context engineering becomes even more critical. An agent that needs to make multiple decisions needs the right information at every step — not just at the beginning.

How It Works

Context engineering involves several key practices: selecting relevant documents and data to include, writing clear system instructions that define behavior, providing few-shot examples that show the desired output format, managing conversation history so the model remembers what matters, and using retrieval systems to pull in the right information dynamically.

A well-engineered context might include a system prompt defining the AI’s role, retrieved documents relevant to the user’s question, conversation history trimmed to the most relevant exchanges, and output format instructions. The key is being intentional about every piece of information the model sees.

Examples

Customer support bot: Instead of just giving the AI a question, context engineering means including the customer’s account history, recent tickets, product documentation, and company tone-of-voice guidelines — producing answers that are specific and on-brand.

Code generation: A developer includes the project’s coding standards, relevant existing code files, and architecture decisions so the AI writes code that actually fits the project instead of generic solutions.

Research assistant: Rather than asking “summarize this topic,” a researcher feeds the AI a curated set of papers, specifies which aspects to focus on, and provides examples of the desired summary format.

Sources

• Simon Willison — Context Engineering
• Anthropic — Building Effective Agents
• OpenAI — Prompt Engineering Guide

Last reviewed: April 2026

You May Also Like

• What Are Agent Handoffs?
• What Are Reasoning Models?
• What Are Guardrails (in AI)?
• What Are Evals (AI Evaluation)?
• AI Glossary: 100+ Terms Every Beginner Needs to Know