Your agent pilot works. Then a model update ships, a colleague edits one prompt, and last month’s invoice-matching results quietly change. Nobody can say which of the pieces around the model caused it, because nobody wrote down what those pieces are.
That is the situation a technical lead faces when an owner asks, “what exactly did we pay for, and can we keep it?” The model is rented. Everything wrapped around it is where your investment lives, and a good share of it should be yours.
For teams that want that wrapper built around their own systems, our forward-deployed AI engineering work is where we do it.
An agent harness is made of six parts, and the business should own and document each one, because together they decide auditability, how easily you can switch models and how much rework your staff absorb. This article walks through the six, what fails in each, and the order in which to build them.
Scope note: for the buy-versus-build side of this question, read custom AI harness vs SaaS copilot.
What an Agent Harness Is
Anthropic’s engineering team (Demystifying evals for AI agents) defines the agent harness (it also calls it the scaffold) as the system that lets a model act as an agent by processing inputs, orchestrating tool calls and returning results. Its guidance adds a consequence: when you evaluate an agent, you are evaluating the harness and the model working together. Swap either one and the result can change.
Vendors use different vocabulary, yet the pieces line up. Context, tools, memory, guardrails, evals and orchestration cover what the sources describe, and the table below is the version we use with clients.
| Part | What it does | What you should own | Typical failure |
|---|---|---|---|
| Context | Decides what the model sees on each step | Instruction files and retrieval rules in plain text | Bloated or stale instructions that raise cost without raising accuracy |
| Tools | Lets the agent read and change real systems | Tool definitions, permissions and the integration code | Vague tool descriptions, so the agent picks the wrong action |
| Memory | Carries state between sessions | The stored files, in a format you can export | Stale notes treated as truth, or sensitive data written to storage |
| Guardrails | Limits what the agent can reach or approve | Policy, sandbox and approval rules | Written rules with no technical control behind them |
| Evals | Scores whether the work was right | Test cases, graders and the history of results | A buggy grader that reports the wrong score with confidence |
| Orchestration | Sequences steps, retries and hand-offs | The workflow definition and its logs | Complexity added before a single agent was shown to fail |
The rest of the article takes these in turn.
Context: Curate What the Model Reads
Models do not read an unlimited amount equally well. Anthropic’s context engineering guidance describes “context rot”: recall degrades as the number of tokens in the window grows, so the aim is the smallest set of information that still gets the job done. Its techniques include retrieving material just in time, compacting long histories, keeping structured notes and delegating subtasks. It also states the costs. Runtime exploration is slower than pre-computed retrieval, and overly aggressive compaction can lose subtle but critical context. What to keep, clear and store before you trim is covered in context engineering for agents.
Instruction files are the most visible form of context. AGENTS.md is the open format for them, and its own site describes it as a README for agents. Practical limits exist: OpenAI’s Codex documentation says it stops adding instruction files once their combined size reaches 32 KiB by default.
The evidence on how much to write is mixed. In a study from ETH Zurich (arXiv 2602.11988), the authors found that context files did not generally improve task success and raised inference cost by over 20% on average. Agents did follow the instructions. Repository overviews, which model providers often recommend, simply did not help. The authors found the files useful for specifying non-standard practices, and developer-written files fared better than model-generated ones.
What this means for you: keep the file short, put only what the agent cannot infer from your code or documents into it, and test it against your own tasks. The business value is lower token spend and fewer odd outputs. Keep the file in your own repository, in plain text, so it moves with you.
Tools: Treat the Interface Like a Product
A tool is any action the agent can take: query the ERP, read a mailbox, create a ticket. Anthropic’s “Building effective agents” guide says tool definitions deserve as much prompt engineering attention as the main prompt, and suggests writing a tool description like a docstring for a junior developer. It also recommends making mistakes harder by design. Its example is that requiring absolute file paths made a model use a tool reliably.
The failure here is quiet. A tool with an ambiguous name or overlapping purpose does not crash. The agent simply picks the wrong one on some fraction of runs. Run many example inputs, read the failures and adjust the definition, as the guide advises.
For ownership, the tool definitions and the integration code belong to you. If they sit inside a vendor’s visual builder, ask for an export before you commit. Our article on MCP and agent tools covers the standard protocol that makes tools portable between agent runtimes.
Memory: Files You Can Read and Delete
Memory is whatever survives between sessions. Anthropic’s documentation for its memory tool states that the tool operates client-side: the model requests file operations and your application runs them. That puts the security work on you. The docs tell developers to validate every path to prevent directory traversal, to cap file sizes, to expire files that have not been accessed for a long time and to strip sensitive data before it is written.
For a business, this is mostly good news. Memory held as files in your storage is inspectable, deletable and exportable, which matters for data protection requests and for audits. It also fails in a predictable way: an old note gets treated as current fact. Give memory files an owner and an expiry rule.
Anthropic reports in an April 2026 post that on BrowseComp-Plus a memory folder lifted Sonnet 4.5 from 60.4% to 67.2%. That is a single vendor run without independent replication, so read it as encouraging rather than settled. Our AI agent memory article covers the design choices in more depth.
Guardrails: Rules Need a Control Behind Them
Guardrails limit what the agent can reach and what it may do without a person approving. Claude Code’s documentation is unusually candid about the limits of its own permission modes. It says the most permissive mode offers no protection against prompt injection or unintended actions, and that auto mode reduces permission prompts without guaranteeing safety. The docs separate two questions: permission modes decide whether the agent asks, while sandboxes and outer isolation decide what an action can reach once it runs.
Anthropic’s sandboxing post says that sandboxing cut permission prompts by 84% in its internal use, and that effective sandboxing needs both filesystem and network isolation. That figure is Anthropic’s own measurement, and the runtime was released as a beta research preview at the time.
A written rule is not a control. A preprint by Ting Yan (arXiv 2608.23550) examined 481 public CLAUDE.md files and estimated that, under the strictest matching standard, only 4.4% of the security rules had a corresponding built-in control. It is a single-author study of public files, and it does not show that the people who wrote those files meant every line to be enforced. The lesson still holds: if the rule says “never touch production”, something other than the prompt should make that true.
What this means for you: give the agent its own credentials with narrow rights, log every action, and require approval for anything irreversible. The auditability that your compliance lead wants comes from those logs and permissions.
Evals: The Asset That Outlives the Model
Evals are the tests that tell you whether the agent did the job. Anthropic’s evals post defines a task, a trial, a grader and an outcome, and it insists the outcome is the final state of the environment (does the booking exist in the database?) rather than what the agent says it did. Its advice for starting is concrete: 20 to 50 simple tasks drawn from real failures is a great start, and evals get harder to build the longer you wait.
Two details from that post matter for managers. First, consistency. A 75% per-trial success rate gives roughly 42% if you need all three of three attempts to succeed. An agent that handles an invoice correctly three times in four is not ready for unattended work. Second, graders can be wrong. The post says scores can mislead if graders are buggy or tasks are ambiguous, and describes a benchmark where grading problems depressed a model’s initial score.
Evals are also the most portable part of the harness. A set of real cases with agreed pass marks works on any model and any runtime. When a vendor ships a new model, you rerun the set and decide in an afternoon. Without it, switching models is a matter of opinion. Our guide to testing and evals for AI agents shows how to build the first set.
Orchestration: Add Structure Only When One Agent Fails
Orchestration decides how steps are sequenced, retried and handed off. Anthropic separates workflows, where predefined code paths direct the model, from agents, where the model directs its own process. It lists five common patterns: prompt chaining, routing, parallelisation, orchestrator-workers and evaluator-optimizer. Its advice is to find the simplest solution possible, because agents bring higher costs and the potential for compounding errors.
Multi-agent designs deserve extra scepticism. In its long-running agent post, Anthropic says it is still unclear whether a single general-purpose coding agent performs best or whether a multi-agent architecture does better. Its April 2026 post reports that, with Opus 4.6, spawning subagents improved BrowseComp results by 2.8% over the best single-agent runs, which is a vendor result on a narrow task. The honest position is that gains from many cooperating agents are unproven, and the extra roles add cost, logs to read and more places to fail.
The Files That Keep Long-Running Agents on Track
A task that takes hours spans many sessions, and each new session starts with no memory of the last. Anthropic’s post on long-running harnesses describes a simple answer built from ordinary files. We add a fifth, which is our own recommendation rather than part of that post:
- A map file (our addition) such as AGENTS.md or CLAUDE.md that tells each new session where things are and what the rules are.
- An init script, which the post describes as an
init.shthat starts the development server and runs a basic end-to-end check. - A JSON feature list where every feature starts as failing. The post reports that the model is less likely to inappropriately change JSON, agents may edit only the
passesfield, and removing or editing tests is declared unacceptable. In the demo it held over 200 features. - A progress log, a text file recording what each session did, so the next one can resume.
- Git history with descriptive commit messages, so bad changes can be reverted.
Work proceeds one feature at a time. The authors say the demo was tuned for full-stack web development, and they only speculate that the lessons carry over to other fields.
These artefacts are worth more to the business than they look. They are the audit trail: what was attempted, what passed, who changed what and when. They are also your exit route. A new model, or a new supplier, can read the same files and carry on.
Build Order and Review Dates
What to build first, and what to skip
The order matters more than the completeness. A sequence that works for a first workflow:
- Pick one narrow workflow and write down what a correct result looks like.
- Build the eval set, starting with 20 cases from real past failures or corrections.
- Define the tools the workflow needs, with narrow permissions and logging.
- Write the shortest useful instruction file, then test whether each section improves your eval score.
- Add guardrails and approvals for anything irreversible, backed by technical controls.
- Add memory when sessions need to continue each other’s work.
- Add orchestration only if the single agent has demonstrably failed.
Skip anything that duplicates what your platform already gives you. Skip multi-agent teams until step 7 forces the question.
Parts go stale, so budget for review
The strongest argument against heavy investment comes from the vendors themselves. Lance Martin of Anthropic writes in a 2026 post that harnesses encode assumptions about what the model cannot do alone, and that these assumptions grow stale as models improve. He cites context resets built for one model generation that became dead weight with the next. LangChain, in the same spirit, writes that as models improve its guardrails will likely become unnecessary. For what a harness-only gain might be worth and a test for deleting rules, see harness engineering.
LangChain’s own result sits in tension with that. In its harness engineering post, changes to the harness alone moved a coding agent from 52.8% to 66.5% on Terminal-Bench 2.0 with the model fixed. That is one vendor, one benchmark, tuned on that benchmark, and LangChain does not separate how much each change contributed. Its authors also warn that changes which overfit one task can cause regressions on others.
Both facts can hold. Harness work pays off now, and some of it will be obsolete after the next model. Hence the advice at the top: own the parts tied to your business (test cases, tool access, logs, written standards) and treat generic scaffolding as disposable. Put a review date on every part, and rerun the eval set whenever the model changes.
If you cannot say who owns each of the six parts today, that is the first finding of any harness review. The cheapest next step is a one-page inventory: for each part, where it lives, who maintains it, and whether you could export it tomorrow.
Frequently asked questions
What is an agent harness?
An agent harness is the code and configuration around a language model that turns it into an agent. Anthropic describes it as the system that lets a model act as an agent by processing inputs, orchestrating tool calls and returning results. In our framing it covers context, tools, memory, guardrails, evals and orchestration, while the model supplies the reasoning.
What is AGENTS.md and do we need one?
AGENTS.md is an open, plain-Markdown format for giving coding agents project instructions, often described as a README for agents. You need one only if you run coding agents on your own repositories. Keep it short, because OpenAI Codex stops reading instruction files at a combined 32 KiB by default, and test whether each line helps.
Which part of the harness should we build first?
Start with a small eval set, then the tools and context for one narrow workflow. Anthropic suggests 20 to 50 simple tasks drawn from real failures as a good start. Without that set you cannot tell whether a later change to context, memory or orchestration helped, so everything else gets built blind.
Do we need multiple agents?
Usually not at the start. Anthropic advises finding the simplest solution possible and warns that agents bring higher cost and the risk of compounding errors. Its own long-running agent post says it is still unclear whether one general agent or a multi-agent setup performs best. Add roles only when a single agent demonstrably fails.
Will better models make the harness unnecessary?
Some of it, yes. LangChain says its guardrails will likely become unnecessary as models improve, and Anthropic says harness assumptions grow stale. The parts that stay useful are the ones tied to your business: test cases, tool access, audit logs and the written definition of a correct result. Review each part whenever you change models.