Context engineering is the discipline of deciding what an agent should read — and what it must refuse
A pilot agent is impressive because it was tuned on a small, curated corpus. A production agent is either dependable or expensive, and the difference is engineered in the runtime: the retrieval parameters, the context budget, the tool contracts, the evaluation harness and the release discipline. This practice builds that layer.
Parameters, not adjectives
Chunk size, overlap, hybrid weighting, rerank depth, score thresholds and recall count are all configurable and all versioned.
A budget per call
Context has a finite size, and assembly is an allocation decision rather than a concatenation step.
Evaluation as infrastructure
A regression set that runs on every release is the only thing that makes a release decision defensible.
Attribution, not vibes
When an answer is wrong, the trace must say whether the cause was retrieval, the tool, the model or the definition.
Prompt engineering asks for better phrasing. Context engineering asks for better decisions.
Prompt work optimises the text sent to a model. Context engineering decides what text exists, in what order, with what authority, and at what cost — and it is a systems discipline that happens to touch retrieval, ranking, assembly, tool contracts and refusal behaviour.
| Dimension | Prompt engineering | Context engineering |
|---|---|---|
| Unit of work | One instruction to one model | A pipeline that decides what a task needs to know before the model is called at all |
| Main lever | Wording, examples, output format specification | Corpus design, chunking, hybrid weighting, rerank depth, score thresholds, recall counts, context budget |
| Failure mode | The model does not follow the instruction | The right information never reached the model, or too much of it did |
| Measured by | Whether the output looks right on a handful of examples | Retrieval recall against a labelled set, answer faithfulness, tool-call correctness, end-to-end task completion |
| Optimisation cycle | Rewrite, try, eyeball | Versioned parameters, regression runs, measured deltas, deliberate promotion or rollback |
| Where it lives | A file in a repository, usually owned by one person | A shared platform capability with a configuration surface, owned by an enablement team |
| Typical end state | A folder of increasingly contradictory instructions | A governed pipeline whose behaviour is reproducible and whose changes are evidenced |
Why this matters for the business rather than the engineering team. Context assembly is where token cost, latency and answer quality are jointly determined. A pipeline that retrieves generously is fast to build, expensive to run and hard to diagnose. A pipeline that retrieves precisely is the opposite on all three counts — and it is the difference between an agent that can be put in front of customers and one that can only be demonstrated to the board.
Context engineering in numbers
Each with parameters, each with a defensible default
A fixed resource to be allocated, not a container to be filled
With protocol boundaries between them
When the answer is wrong, the trace must say why
Prompt engineering asks for better phrasing. Context engineering asks for better decisions.
Both are real skills. Only one of them survives contact with an enterprise corpus, a changing policy set and a regulator asking why the agent said that.
- Optimises the phrasing of the instruction
- Iterates on wording until output looks right
- Evaluated by reading the output
- Improves with a skilled individual
- Degrades silently when the context changes
- Cannot explain why it worked on Tuesday and not Wednesday
- Optimises what the agent knows when it decides
- Treats retrieval, ontology, granularity and freshness as design parameters
- Evaluated against a test set with a stated threshold
- Improves with a systematic method anyone can repeat
- Detects and attributes degradation when the corpus changes
- Can explain which stage failed and what would fix it
Every stage has a parameter, and every parameter has a defensible default
Retrieval quality is not a setting you turn once. It is a set of coupled decisions that respond to each other and to the corpus. We configure them explicitly, record the configuration with each release, and re-tune them when the corpus or the task changes.
| Parameter | What it controls | Defensible starting point | When and why to move it |
|---|---|---|---|
| chunk_size | The token length of a retrieved passage | Around 400–600 tokens for narrative text; larger for tabular and contractual content | Move up when answers need whole clauses; move down when passages are noisy and the task is narrow and precise. |
| chunk_overlap | How much adjacent content is repeated between chunks | Around 10–15% of chunk size | Increase when a fact routinely straddles a boundary and is being split; decrease when repetition is biasing the ranking. |
| table_min_retain | The minimum number of characters of a table kept intact with its header row | Keep the full table and its header when the table fits the chunk, otherwise keep the header and the referenced rows | Never let a table be chunked row-wise without its header — a chunk without its header is unusable context and pollutes the candidate set. |
| hybrid_weight | The blend between lexical and semantic retrieval | Balanced, with lexical weight higher for identifiers, part numbers and error codes | Shift toward lexical where queries contain exact strings; shift toward semantic where users describe rather than specify. |
| semantic_boost | How much a semantic score is allowed to outweigh an exact match | Limited, and capped, so an exact identifier match cannot be outranked by a merely similar passage | Any business where a wrong entity match is expensive. This is the parameter that stops a confidently wrong answer about a real customer. |
| first_pass_k | How many candidates the first retrieval stage returns | A few hundred, generous | Increase for large corpora with long-tail queries; the cost is paid in the rerank stage, not in the model call. |
| rerank_top_n | How many candidates survive into the rerank stage | Between 20 and 50 | Reranking is the accuracy step and the latency step. Move it up only when measurement shows recall is the binding constraint. |
| score_threshold | The minimum relevance score for a passage to be included at all | Set from a labelled sample, and tuned rather than guessed | Set it generously high and the agent abstains too often; set it low and noise floods the context. This single number trades coverage against precision and should be measured, not tuned by feel. |
| recall_count | How many passages are finally assembled into the context | As many as the budget allows after ranking, and no more | Never increase recall without re-checking the context budget; the two are the same decision viewed from two ends. |
Measure recall before you tune ranking
Build a labelled set of real questions with the passages that should answer them. If the correct passage is not in the candidate set, no reranker will save you — the fix is upstream in chunking and lexical coverage.
One configuration per corpus
Contracts, policies, tickets and reference tables need different chunking. A single global configuration is the most common reason a "good" pipeline works on one document type and fails on another.
Configuration is versioned with the agent
A regression run that passes under a changed chunk size is not evidence of anything. Parameters are part of the release artifact and are restored exactly when you roll back.
Every stage has a parameter, and every parameter has a defensible default
Six stages. The expertise is in setting them per document class and knowing when to move one. The defaults are defensible; the defaults are not always correct for your corpus.
Ingest
Sources, permissions and effective dates captured at the point of ingestion — not inferred later.
Parse and structure
Headings, clauses, tables and exceptions preserved as structure rather than flattened to text.
Chunk
Granularity set per document class. A policy and a contract want different chunk sizes, and using one setting for both is a common and expensive error.
Embed and index
With version and lineage recorded, so an answer can name the document version it relied on.
Retrieve and filter
Permission and freshness filters applied before ranking, not after.
Rerank and assemble
Within an explicit context budget. What does not fit is a decision, not an accident.
A context budget is a fixed resource that must be allocated, not filled
Every call has a ceiling on how much context it can carry. Treating that as a container to fill produces the most common production failure: a context that is technically relevant, structurally overwhelming and behaviourally worse than a smaller one.
Budget by role, not by pile
The context is divided into named segments with their own allocation: task instruction, retrieved knowledge, business definitions, conversation state, tool results, output format. Each segment has a ceiling, and a segment that overruns is truncated by its own rule rather than by a global cut.
Trim by relevance, then by structure
Within a segment, passages are dropped lowest-relevance-first; passages that are part of a single required structure — a table with its header, a contract with its clause — are dropped or kept together.
Keep the citation spine
Every retained passage carries an identifier and a location, and the identifier is what the answer cites. A trimmed passage that leaves a dangling claim in the answer is worse than a smaller, fully attributable answer.
Design for refusal
When the budget cannot hold enough relevant material to answer responsibly, the correct output is a refusal with a pointer to a human — not a thinner, more speculative answer. Refusal is a designed behaviour, and it is evaluated like any other behaviour.
Segment budget, worked example
Illustrative allocation for a customer-operations agent answering a policy question. The proportions matter; the absolute values depend on the model's context window and the task.
| Context segment | Share of budget | Assembly rule |
|---|---|---|
| Task instruction and role | Small, fixed | Never trimmed. If it does not fit, the configuration is wrong. |
| Business definitions | Fixed, semantic | Injected from the semantic layer, not retrieved. These are the rules the agent must not reinterpret. |
| Retrieved knowledge | Largest, variable | Ranked passages, trimmed lowest-relevance-first, structured units kept intact. |
| Case and conversation state | Fixed, growing | Summarised, with the summary itself cited back to the events it replaces. |
| Tool results | Variable, capped | Per-call caps, structured field selection, oversized payloads rejected at the tool boundary rather than truncated silently. |
| Output contract | Fixed | Format specification and required citation fields. |
A context budget is a fixed resource that must be allocated, not filled
Once an agent can see more than it can attend to, quality degrades silently and nobody notices until a decision goes wrong. Allocating the budget is the discipline; filling it is the anti-pattern.
Four classes of tool, and the protocol boundaries between them
An agent's reach is the sum of what it may call and what those calls may change. Most programmes flatten that into a single list of functions, and then discover that the security review, the cost model and the blast radius all depend on which list a tool happens to be in.
Read-only
Retrieval, lookup, query, search. No state change, so the risk is disclosure rather than corruption. The main control is the permission set of the identity the agent runs under, which is why that identity is scoped to the task and not to the person who happened to ask.
Drafting
Produces content for a human to review: a draft, a proposed reply, a summary, a filled template. Low blast radius because a person is in the loop by construction. The measure to watch is the rate at which humans approve without reading, which is where the value quietly disappears.
Transactional
Writes to a system of record: creating a case, issuing a credit, updating an order, submitting a payment. Here the write itself is the risk, and the controls are idempotency, a compensating action, and a pre-write validation against the semantic layer.
Irreversible
Cannot be undone: a payment, a termination, a disclosure, a regulatory filing, a message sent to a customer. These are excluded by default and admitted only with a named owner, a verified pre-condition, a two-step confirmation and a permanent record of what was decided and why.
How the layers connect
Tools are exposed to an agent through a contract that declares the entity it acts on, the parameters and their ranges, the states in which it is legal, and the class it belongs to. Two integration styles sit on top of that: MCP for tool and resource exchange, where the agent-side and the business-side declare capabilities in a standard form rather than in bespoke glue; and A2A for agent-to-agent collaboration, where an agent delegates a task to a peer with its own mandate and authority rather than reaching into its internals.
Underneath both, ordinary REST API and webhook integrations carry the long tail of systems that will never speak a modern protocol. The discipline is identical in all three cases: the interface is described, versioned, permissioned and logged, and the agent never holds a credential broader than the capability it was granted.
Read and write are separated on purpose. The most common serious incident pattern in enterprise agent deployments is an agent with legitimate read access to a customer record and legitimate write access to a messaging system, combined into one credential. Capability separation makes that combination structurally difficult rather than a matter of policy.
What every exposed operation declares
Target entity and action
What it acts on, in the business's own vocabulary, from the semantic layer.
Typed parameters with envelopes
Ranges, allowed values and required fields, enforced at the boundary rather than described in prose.
Legality preconditions
The states and rules under which the call is valid, so an illegal call fails fast.
Idempotency and compensation
Whether a repeat call is safe, and what undoes the action if it was wrong.
Tool class and required approval
Which of the four classes it belongs to, and therefore the approval and logging it attracts.
Four questions we ask before an agent goes live
If the right passage is not retrieved, does the agent say so, or does it answer from the model's general knowledge? The refusal path has to be designed and tested, because it is rarely exercised by accident and never by luck.
A partial failure is the normal case, not the edge case. The agent must distinguish "the tool said no" from "the tool did not answer" and from "the tool answered and it was wrong", and the three need different handling.
Parameters, model, corpus version and prompt version are attached to the execution record. If they are not, a regression cannot be reproduced and a bad answer cannot be diagnosed.
Enumerate what a single wrong tool call would actually change, and confirm that the write path is idempotent, compensatable, or restricted enough that a mistake is recoverable at a known cost.
Four classes of tool, and the protocol boundaries between them
Most estates have only the first class and reach for the fourth when they need the third. Building them in order is slower and much cheaper.
Read tools
Read-only access to systems of record. The safest class, and where most integrations should start. The agent can be wrong here without consequence, provided the escalation rule works.
Write tools
Change the state of a system. Every write needs an idempotency key, a reversal path and a named approver — or it does not go in.
Commit tools
Transactions that cannot be rolled back: payments, filings, customer communications. These need full human approval, every time, with the evidence shown at the point of decision.
Authority tools
The tools that change who can do what. These are the ones to build last, review most, and never expose to an agent that has not been in production for a full operating cycle.
Model choice is a deployment decision, not a brand decision
Enterprises end up running several models at once: a large general model for hard reasoning, a small fast one for classification and routing, an embedding model for retrieval, and a locally hosted model where data cannot leave the network. The runtime has to manage all of them without the application knowing which is which.
| Runtime concern | What it must support | What it must not do |
|---|---|---|
| Multi-model management | Route by task and cost, run several models concurrently, and change the route without redeploying the agent | Require the application to be written against one vendor's interface |
| Private inference | Host a model inside the customer's network or a dedicated environment for data that cannot be transmitted | Treat "the data is in our cloud" as an acceptable answer to a residency requirement |
| Hybrid deployment | Split one workload across shared and private capacity, with a defined rule for which requests go where | Make the split implicit, so nobody can say where a given piece of data was processed |
| Compute elasticity | Scale for batch evaluation and peak traffic, and shed load gracefully rather than failing on the first burst | Let a spike in one agent's traffic degrade every other agent on the same platform |
| Key management | Hold credentials outside the agent's own context, rotate them, and scope them to a single capability | Put a shared service credential into a prompt, a config file or a tool description |
| Sandboxed execution | Run generated code in an isolated environment with a restricted file system and no ambient network reach | Execute agent-generated code in the same trust zone as the application |
Cost per completed task, not per token
A cheap model that causes two retries and a human escalation is more expensive than an expensive model that answers once. The unit that matters is the cost of the completed task including supervision.
Budget the critical path
In a customer-facing flow, the retrieval, rerank and tool round trips are usually the latency. Each is a decision with a measured cost, and the budget is set end to end rather than per component.
One platform team, many agents
The runtime, the context pipeline and the evaluation harness are shared capabilities with one owner. Business teams consume them; they do not each run their own stack.
Four steps, four kinds of assessor, one regression set
Evaluation is not a final gate before launch. It is a standing capability that runs on every change, on live traffic, and produces a number that can be compared with the number from last week.
The four-step method is fixed. Build a representative test set from real cases, including the difficult ones and the ones that have gone wrong before. Run it against a version. Score it with assessors of different kinds. Compare with the current production baseline and decide — promote, hold or roll back. The steps are simple; the discipline of never skipping them is what makes it work.
- Test set — real cases, real volumes, with the expected outcome written down by the business, not generated from the model's own previous answer.
- Execution — the same set, the same configuration, every time. Determinism of the harness matters more than sophistication of the scoring.
- Scoring — multiple assessor types, because no single kind sees the whole failure.
- Comparison — against the live baseline, on business measures as well as technical ones, with an explicit promotion decision recorded.
The regression set is the asset. It is the one artefact that makes tomorrow's change safer than today's. We build it from real cases, including the exceptions, and we keep growing it from every bad case the agents hit in production.
What each one can and cannot see
| Assessor | Catches | Blind to |
|---|---|---|
| Rule-based | Format, required fields, citations present, prohibited content, numeric agreement with the source record | Whether the answer is relevant or true in substance |
| Model-judged | Relevance, faithfulness to the supplied context, tone, completeness against the task | Facts the context did not contain; it will not notice a missing source |
| Reference-based | Agreement with a human-written expected answer, graded for partial credit | Cases where more than one answer is acceptable |
| Human review | Business correctness, appropriateness, anything a rule and a model both miss | Scale and cost — which is why it samples rather than covers |
Human review samples by risk, not uniformly. Low-risk drafting cases are sampled lightly; transactional and irreversible cases are reviewed closely. Spending a fixed human budget across everything is how review becomes either unaffordable or useless.
Four steps, four kinds of assessor, one regression set
This is the difference between a context pipeline you can operate and one you are afraid to touch.
Assemble the set
From your own historical cases, weighted towards the hard ones and the ones that escalated.
Declare the assessors
Rule-based where a rule exists, model-as-judge where judgement is required, human where stakes justify it. Each with a stated boundary.
Run against live traffic
Every conversation assessed. Failures clustered by the stage that caused them.
Gate every change
Context, prompt, model or tool change re-runs the suite. No change ships on a hunch.
When the answer is wrong, the trace must say why
"It hallucinated" is not a diagnosis. A production agent's trace should let an engineer answer four questions without opening a log aggregator: was the right information retrieved, was the right tool called, was the definition right, and was the output constrained correctly.
Full request tracing
Every agent run records the instruction, the segments assembled, each retrieved passage with its score and provenance, each tool call with its arguments and result, the definition version applied, and the final output. This is the raw material for everything below.
Bad-case attribution
Each failure is classified into a cause: retrieval miss, wrong ranking, missing knowledge, wrong tool, wrong definition, genuine model limitation. The distribution is reviewed monthly, because the top cause determines the next piece of engineering work — and it is rarely the model.
Cost attribution
Inference, retrieval, rerank and tool cost attributed per agent, per task type and per business case, including the cost of human review and escalation. Cost per completed task, attributed to a business outcome.
The data flywheel
Every confirmed bad case becomes a regression case the same week. The evaluation set grows from production, which means quality improves for the reasons that actually matter to the business rather than for the reasons that are convenient to measure.
Attribution table
The mapping from symptom to the layer that caused it. This is the artefact that turns an argument about the model into an assignment of work.
| Symptom | Likely cause | First thing to check |
|---|---|---|
| Answer contradicts the policy | Wrong or outdated passage retrieved | Corpus freshness, chunk boundaries, score threshold |
| Confidently wrong about a specific customer | Entity resolution failure | Identity rules, semantic boost cap, record visibility |
| Correct answer, wrong action | Tool contract or state precondition | Legality checks, permitted state machine, tool arguments |
| Inconsistent between runs | Assembly non-determinism | Tie-breaking, retrieval order, model temperature settings |
| Degrades as context grows | Budget allocation | Segment shares, trimming order, retrieval count |
When the answer is wrong, the trace must say why
Seven questions. If your instrumentation cannot answer all seven, you can observe the agent but you cannot operate it — and the difference shows up during your first incident.
| Question asked after a wrong answer | What the trace must show |
|---|---|
| Which sources did it use? | The exact documents and records retrieved, with versions |
| Did it retrieve correctly? | The candidate list, the scores, and what was filtered and why |
| Did the right thing survive? | The reranked result and where the correct passage was ranked |
| Did it assemble correctly? | What made it into the final context and what the budget excluded |
| Did it reason correctly? | The reasoning trace, not just the conclusion |
| Should it have refused? | The confidence level, the escalation rule, and whether it triggered |
| Who approved it? | The named approver, or the record that no approval was required |
What we do not promise
Context engineering is where vendors make the most extravagant claims. We would rather be precise about what is achievable, because the difference matters when something goes wrong in production.
- We do not promise zero hallucination. No retrieval design eliminates the possibility of an unsupported statement. What we commit to is narrower and more useful: answers that are attributable, that cite their sources, that are evaluated against a regression set before release, and that are traceable back to the exact configuration that produced them.
- We do not promise a fully autonomous agent from a data and configuration exercise. Context quality is necessary and not sufficient. Autonomy additionally requires evaluation maturity, a supervised operating model and governance — which is why this practice sits inside a wider set rather than standing alone.
- We do not take on the data engineering backlog as part of an infrastructure engagement. If the corpus is missing, stale or inconsistent, that is a separate programme with its own owners. We will identify it and say so clearly rather than absorb it into an infrastructure scope and deliver nothing.
- We do not lock you into a single retrieval stack. The parameters, the evaluation harness and the trace format are yours, documented and portable. If a different engine serves a corpus better later, moving should be a configuration change, not a rebuild.
Three properties of the system we leave behind
Measurable
A regression set with a score, a baseline and a comparison for every change, and a named owner for the number.
Attributable
A trace that identifies the layer responsible for any failure, so the next piece of work is chosen from evidence rather than instinct.
Reversible
Versioned parameters, a rollback to a known configuration, and the ability to reduce an agent's authority without redeploying anything.
Where this practice connects
Knowledge & context capability
The platform-side view of retrieval, knowledge bases and permission traceability.
Business semantics
Where the definitions injected into the context segment come from.
Governance
Where the trace becomes audit evidence and the tool classes become controls.
What we do not promise
We do not promise a retrieval accuracy figure before we have seen your corpus. We will not tell you an agent is grounding on your documents when it is grounding on a cached summary of them. And we will not rebuild your entire corpus in phase one when a granularity change would fix most of the problem for a fraction of the cost.
What this page answers
Every topic below is covered in the sections above, and each links to the page that goes deeper on it.
Bring us one agent that is good in the demo and disappointing in production
A 45-minute working session with a platform engineer. We will ask for the trace of one failed run and read it together, then tell you which layer we think is responsible. If the answer is the model, we will say so.