The Second Half of AI Coding Is About the Harness, Not the Model

Humans steer. Agents execute.

Over the past few months, I have worked through much of what OpenAI, Anthropic, LangChain, Thoughtworks, and HumanLayer have written about agents, alongside practical accounts, open-source projects, and enterprise retrospectives. The more I read, the more I feel that many people’s understanding of AI coding remains at an early stage.

Our favorite topics are still models, prompts, MCP, and context windows, as though familiarity with those terms meant we understood AI programming. Yet when you examine what experienced teams actually do, much of their engineering effort goes into the machinery surrounding the model.

People using the same Claude Code, Codex, or Cursor get strikingly different results. Some agents lose direction after two file changes or introduce three new bugs while fixing one. Others inspect repositories, run tests, read logs, open pull requests, respond to reviews, repair regressions, and keep working for hours. It is tempting to attribute this to model quality. Look closely, though, and the difference often lies in the working environment around the model.

That environment is acquiring a more established name: the harness.

At first, I disliked the word. It sounded like another old idea in new packaging. More reading changed my mind. The discussion goes beyond clever prompts, connecting MCP servers, or opening extra agents. It asks a harder question:

Have you given AI an environment in which it can work continuously, correct itself, and accumulate knowledge over time?

Without that environment, even a strong model offers only occasional flashes of brilliance. With it, a chat box may become a collaborator that actually executes. To put it bluntly, many people have never given AI a suitable place to work.

What Is a Harness?

LangChain’s “The Anatomy of an Agent Harness” offers a concise formula: Agent = Model + Harness. It cuts through a great deal of vague discussion. The model determines the capability ceiling; the harness establishes a reliable floor. The model determines what is possible, while the harness determines how consistently those possibilities become real results.

A harness is, in plain terms, the AI’s workbench. It includes a filesystem, an understandable repository structure, readable documentation, tools it can call, access to runtime results, and feedback that distinguishes success from failure. Beyond that, it must support handoffs between sessions, recovery, clean context, and limits on the spread of bad patterns through the repository.

Without these, even an impressive model is a probabilistic machine that talks. With them, it can begin handling genuinely complex, long-running work.

An analogy from an Alibaba article makes the term especially apt. The model resembles a powerful, energetic horse whose behavior is partly opaque. The harness includes reins, saddle, and protective equipment. The human engineer is the rider, choosing a direction, designing the environment, setting constraints, and responding to feedback. Neither horse nor rider alone is enough. Their engineered connection determines how far the system can travel.

A Tencent article cites Mitchell Hashimoto’s straightforward definition: whenever an agent makes a mistake, invest in an engineering solution that prevents the same mistake next time. This brings an abstract methodology back to an everyday action. Instead of repeating corrections, turn them into the system. A rule in AGENTS.md, a script, a screenshot tool, or a linter rule all serve that purpose. A harness is often built one failure at a time.

From Prompt Engineering to Context Engineering to Harness Engineering

AI engineering has followed a clear progression.

First came prompt engineering: how should we talk to the model? Few-shot examples, chain-of-thought, role assignment, and output constraints could be useful. We refined prompts inside a fixed conversation window to elicit better answers from a black box.

Then came context engineering: what should the model see? Knowledge bases, tools, memory, retrieval, file selection, and compression moved the discussion beyond conversational technique into the design of model inputs. Karpathy’s observation that context engineering matters more than prompt engineering rings true to me. A model often struggles because it has not seen the relevant information.

Today, deciding what it sees is still insufficient. We also need to decide how the whole environment operates:

Within what boundaries does this agent work? How does it obtain tools, verify completion, recover, retain knowledge, and prevent growing disorder in the repository?

That is harness engineering.

Prompt engineering has not disappeared, but its position has changed. Once, being good at prompts was treated as being good at AI. Prompts now look more like one component in a machine. If you study only that component and neglect the environment, constraints, feedback, and recovery, you get a familiar outcome: AI occasionally dazzles, then loses coherence over a longer workflow, becomes confused by real systems, and starts leaking at every seam in a complex project.

OpenAI’s Real Message Was a Change in the Engineer’s Role

OpenAI’s “Harness engineering: leveraging Codex in an agent-first world” helped popularize the term through a case nobody could easily ignore. Three engineers, five months, no handwritten code, and a real product of roughly a million lines serving internal users and external alpha testers. The article describes around 1,500 pull requests in those five months, averaging 3.5 per engineer per day, with throughput continuing to grow as the team expanded from three to seven.

The numbers are less important than the repeated message: the initial bottleneck was an incomplete environment. Early progress was slow because the repository lacked sufficiently clear tools, abstractions, structure, and constraints. Codex could write code, but could not reliably advance toward high-level goals. Engineers gradually shifted from implementing features themselves to equipping the agent with missing capabilities.

That point challenges a common reflex. When AI output disappoints, people blame the model, insufficiently detailed prompts, or unfamiliar training data. OpenAI offers another diagnosis: when an agent stalls, ask what the system lacks. Often the working environment is the weak link.

Once you adopt that perspective, your actions change.

For example, the team initially packed system explanations, architectural standards, code style, and cautions into a huge AGENTS.md. More information seemed better, but it overwhelmed the agent. They later shortened the file into an entry point and map, moving detailed designs, architecture, plans, and quality documents into separate repository files for retrieval when needed.

This was more than tidying documentation. It reflected a clear judgment: the agent does not need everything immediately; it needs the right information at the right time. A map is more useful than an enormous manual in which the important page is difficult to find.

The team also let Codex do much more than read source. It could run Bash and code, open the application, drive a browser, inspect DOM snapshots and screenshots, and query logs, metrics, and traces. Together, these form a real workbench. Many bugs appear only at runtime. Without that feedback, the agent guesses blindly; with it, the agent can implement, run, observe, correct, and verify.

Another important step was turning written suggestions into mechanical constraints. They enforced a dependency direction such as Types -> Config -> Repo -> Service -> Runtime -> UI within business domains using custom lint rules and structural tests. Stable boundaries can matter more than freedom to improvise. Clear constraints often make agents more dependable.

One detail I particularly liked was redesigning linter errors for agents as well as humans. We used to assume humans were the primary readers of every tool. Increasingly, the first reader may be an agent. Much tacit engineering knowledge therefore needs translation into forms machines can interpret consistently.

The team also found that bad patterns spread. Agents imitate existing repository conventions, including poor ones. Over time, weak abstractions, habits, and code smells propagate like mold. Weekly human cleanup was unsustainable. Instead, they recorded “golden principles” and introduced recurring checks of documentation and code patterns, with repair PRs for outdated guidance and drift.

At that point, a harness is more than a few tools. It becomes an engineering mechanism for maintaining order, limiting entropy, and preventing long-term decay.

What impressed me most was the redefinition of the engineer’s job. Humans spend less time directly implementing and more time designing structures, defining rules, arranging feedback, and maintaining order. Software engineering itself is being reorganized around collaboration with agents.

Anthropic: Long Tasks Fail through Forgetfulness and Premature Confidence

OpenAI makes the broad direction vivid; Anthropic’s articles feel more like an inspection of the job site. “Effective harnesses for long-running agents” and “Harness design for long-running application development” explain in grounded terms why extended tasks fall apart.

The first problem is familiar: work crossing multiple context windows loses memory. The next session may not know what the previous one understood, and may fall into the same traps. The project accumulates half-built features, half-completed fixes, and partial understanding until nobody can clearly describe what remains blocked.

The second problem is premature completion. Halfway through, an agent glances at a page, decides it seems usable, and starts wrapping up. It resembles an overconfident intern. Anthropic’s diagnosis is that weak state management and verification loops matter as much as reasoning ability. The work becomes scattered.

Their response is practical. An initializer first sets up the environment, creates progress records, decomposes work, and leaves handoff artifacts. Later coding sessions advance a clear subtask and leave a usable handoff. Each session inherits a recorded state instead of reconstructing the world from scratch. The central goal is not forgetting.

Anthropic also emphasizes end-to-end verification. Models readily confuse “I changed the code” with “the feature works.” Passing a unit test or calling an endpoint with curl may look sufficient, while the actual user path contains a broken page, inactive button, or disconnected interaction. Browser automation and testing from the user’s perspective expose what a diff cannot.

A further step separates generation and evaluation. The agent producing the implementation does not grade itself. Another agent operates the page with tools, checks features, captures screenshots, compares against criteria, and reports defects. The useful insight is not the sophistication of multiple agents. It is an uncomfortable truth:

Model self-evaluation is often unreliable.

In subjective work such as design and frontend implementation, a model may struggle to distinguish merely presentable from genuinely good, or barely functional from smooth. It readily praises a half-finished result. Again, an intern—only a particularly stubborn one.

Anthropic’s writing on context engineering, sandboxing, and tool design addresses related problems. Context is an attention budget. Sandboxes make autonomy possible. Clear tools matter more than numerous tools. Together, these changes turn an improvisational intelligence into an engineering participant that works reliably within boundaries.

A Tencent article puts it well: discipline has shifted from writing code well to building the structures that let agents work well. Discipline increasingly appears in documentation organization, mechanically enforced constraints, feedback loops, early acceptance criteria, and exposing runtime results. Restricting engineering discipline to elegant individual lines of code misses this change.

Four Cases Make the Idea Concrete

A discussion confined to concepts can make a harness sound grandiose. Specific cases clarify what it provides.

1. The Editing Interface Is Part of the Harness

An Alibaba article discusses independent developer Can Duruk’s observation that many coding agents fail because their editing interface is fragile. Producing patches, precisely reproducing old text, or following specialized diff syntax can force models to express their intent awkwardly.

Hashline attaches short hash labels to file lines so the model can target labels rather than reproduce whole passages. The same models then achieved substantially higher success rates. I like this example because it highlights something easily missed:

Even a low-level interaction such as editing a file belongs to the harness.

What looks like a test of the model may actually be a test of the interface between model and environment.

2. Technical Debt Can Replicate Faster with Agents

A developer’s retrospective makes another painful point. A human may recognize ugly code as historical baggage to avoid imitating. An agent may simply copy what it sees. A temporary compromise, hardcoded value, or cross-layer call can become an apparently legitimate pattern and spread into later work.

This matches OpenAI’s experience. Technical debt in human teams often accumulates gradually; with agents, it can resemble a self-replicating virus. The prevailing style in a repository gets amplified.

Teams therefore increasingly invest in repository hygiene. They encode judgment into lint rules, structural tests, templates, scripts, inspections, and background refactoring PRs instead of relying on weekly manual cleanup. Ultimately, a harness confronts an old question:

Can you turn engineering taste from informal experience into executable rules?

3. Subagents as Context Firewalls

HumanLayer describes a subagent as a context firewall, an analogy I find accurate.

If every search, exploration, grep, read, failed attempt, and log enters the main context, a complex task soon drowns in temporary noise. A subagent can solve a local problem in its own context, then return a compressed conclusion with references. That protects the main thread.

The value of multiple agents often comes from:

Isolating noise.
Compressing information.
Preserving the main thread’s sense of direction.

More agents are not inherently more advanced. Reliable handoffs, compression, and recovery are the goal.

4. Quiet Success, Actionable Failure

The Alibaba article also discusses HumanLayer and LangChain practices that ordinary teams can adopt. HumanLayer initially returned complete test output after every change. That flooded the context rather than improving reliability.

Their resulting principle is simple:

Success should be quiet; failure should speak.

The agent needs feedback without drowning in it. Successful builds, tests, lint, screenshots, and logs usually require only a concise signal. Failures should return the smallest actionable error. This protects the context budget.

LangChain adds pre-completion checklists and loop detection: one challenges whether the work is truly complete, and the other detects repeated circling. These mechanisms may not look like model capabilities, but they prevent common mistakes.

HumanLayer Makes the Small Engineering Details Explicit

Where OpenAI and Anthropic explain broad directions, HumanLayer’s work often reads like field notes from people who have encountered the problems firsthand.

I especially agree with its restrained approach to CLAUDE.md. Such a file has substantial influence because it enters each session, but it can easily become a dumping ground. Trying to put everything in your head into the file produces pollution. Short, stable guidance covering recurring rules the model cannot infer is often more effective.

That is the same reasoning behind OpenAI’s decision to split up its oversized AGENTS.md.

HumanLayer also attributes many coding-agent problems to the configuration surface. Skills, MCP, memory, subagents, and hooks constitute the agent’s working configuration. Many people have never supplied a coherent one. They describe their tool as an agent while using it like a disposable chat box.

I appreciate that these accounts do not pretend the process is naturally elegant. Models forget, become confused, make confident errors, drown in noise, and struggle against poor interfaces. The engineering task is to close those gaps one by one. A harness often grows through repairing leaks rather than adding decoration.

Thoughtworks and Inngest: System Discipline Matters More than Framework Hype

Many of these articles eventually converge on constraints and system discipline.

Thoughtworks’ “Harness Engineering” brings the discussion back to engineering instead of getting carried away by terminology. Its three themes—context engineering, architectural constraints, and entropy control—closely match OpenAI’s experience. Practicing teams are converging on the same problems.

Inngest’s “Your Agent Needs a Harness, Not a Framework” approaches from another direction: many agent failures are infrastructure problems. What happens when step five fails? How do simultaneous messages avoid corrupting state? How do you divide a long loop into retryable steps, persist intermediate results, transfer control, or send notifications after failure?

Traditional software engineers consider these ordinary questions. Yet when building agents, many revert to trying another prompt.

I increasingly agree with Inngest: teams often lack a harness more than a framework. The difficult work is frequently keeping the whole system controlled and recoverable.

Specifications, Evaluation, and Observability

Another easily neglected layer is spec, eval, and observability.

Unclear specifications allow agents to move efficiently in the wrong direction. OpenAI keeps designs, architecture, plans, and quality documents in the repository. Anthropic structures feature lists. GitHub’s spec-kit turns related ideas into a product. All address the same questions: what goal is the agent pursuing, through what process, and when is the work finished?

Across sessions, steps, and agents, vague specifications become much more expensive. Humans can sometimes compensate with tacit knowledge; agents cannot reliably do so.

Evaluation follows the same logic. OpenAI’s material on eval skills, agent evals, and trace grading evaluates the process rather than merely whether the final answer sounds plausible. Agents may go wrong at an intermediate step. Without traces and task-level evaluation, many failures remain invisible.

LangChain’s “Improving Deep Agents with harness engineering” offers a persuasive example: its coding agent moved from the top 30 to the top five on Terminal-Bench 2.0 by changing the harness, particularly self-verification and tracing, rather than the model. System design can produce substantial performance gains.

Observability is equally important. OpenAI exposes logs, metrics, and traces to Codex. Anthropic’s evaluators actually operate the application in a browser. If agents cannot see runtime feedback, they can only guess. Guessing occasionally works, but rarely supports consistent delivery.

Stronger Harnesses Make Specifications More Important

As the term becomes popular, some people wonder whether better environments and tools will reduce the need for specifications, design documents, and behavioral constraints. If agents become more capable, can they simply inspect the repository and work everything out?

That appealing argument gets the relationship backward.

OpenAI makes a hard point: from an agent’s perspective, information unavailable at runtime effectively does not exist. Knowledge in Google Docs, Slack, meeting notes, or someone’s head is invisible. It can reason from information that is explicitly available, versioned, and actually read.

A harness can amplify only what it can access. Clear standards, contracts, and design decisions become more consistent output. If those definitions are missing or vague, the harness amplifies guesses instead.

That is why I agree with the Tencent article that harness engineering and specification-driven development are two layers of the same undertaking.

The harness defines the working environment. The specification defines what that environment says, so the agent can reason, execute, and verify. The harness is an amplifier; the spec is its input signal. A stronger amplifier makes input quality more consequential. A more powerful engine needs better navigation. Guardrails matter more as the vehicle gets faster.

Tencent describes three roles for a spec. First, it is a map for reasoning. Code shows what exists, but may not explain why it was designed that way, which boundaries are fixed, or what a field means across services. Second, a spec carries semantic constraints. Lint checks formatting and structural tests check dependencies, but business meaning, error-code contracts, and state transitions often escape mechanical checks. Third, the spec defines correctness for the feedback loop. Verification, regression testing, scoring, and trace grading need an explicit description of what should happen.

These may be the weakest parts of many teams’ systems. Repositories contain plenty of code but lack a high-level view, clear boundaries, or navigation. Agents enter a maze. Semantic constraints are even easier to neglect: teams assume lint, types, and tests are enough, while serious problems hide in cross-service contracts, business meanings, error semantics, and transition conditions that “everybody knows.” Human colleagues compensate through shared context. An agent may silently invent a plausible answer and implement it.

Tencent gives a representative example: service A defines an error code, but the agent implementing service B does not know its cross-service meaning and guesses how to handle it. The important semantic definition never entered the spec. The failure comes from missing definitions, not simply inadequate model ability.

Harness engineering therefore amplifies one fact: what is explicitly recorded in the repository matters far more than before. Experienced engineers once compensated for mediocre documentation. As agents become the main executors, knowledge hidden in conversations, group chats, and personal memory must become explicit, or the weakest parts of the system will operate on guesses.

Mature harnesses make specifications more valuable. They amplify both the good and the bad in a repository: clarity when you provide clarity, ambiguity when you provide ambiguity.

Tencent also explains why a huge AGENTS.md is a trap. Execution constraints and semantic constraints are different. “Do not call this API,” “keep files below this length,” and “do not run this command” describe actions. “What does this field mean?”, “how does this state change across services?”, and “what does this error represent in this context?” describe semantics. Combining everything into one growing document can undermine both. The agent loses the entry point and the important meaning.

Not everything belongs in an agent instruction file. AGENTS.md should be a directory and action entry point; specifications should define system semantics and behavior. Forcing them together creates a manual that neither humans nor agents read well.

Another valuable point is specification drift. Code drift often announces itself through test failures, type errors, or broken behavior. Specification drift is quieter. A document remains labeled “Active” even after the system has evolved beyond it. The next agent then builds against an obsolete map. In an agent-driven environment, outdated documentation can actively mislead.

Tencent’s conclusions fit naturally into harness engineering. First, scarce human attention means review should move earlier toward the spec. Second, action entry points and semantic definitions should be managed separately. Third, spec drift is inevitable and eventually requires active detection rather than manual care alone. Fourth, a failure should prompt us to ask what capability the AI lacks to do the work correctly. Together, these explain why SDD becomes more significant in the harness era.

Benchmarks Increasingly Evaluate the Whole Harness

Consider SWE-bench Verified, OSWorld, GAIA, AppWorld, WebArena, WorkArena, Terminal-Bench, and HAL. The trend is clear: meaningful benchmarks increasingly assess extended, closed-loop work instead of single-turn answers.

Can the agent operate a terminal reliably, complete real browser tasks, avoid corrupting a multi-step environment, and preserve direction across tools, states, and context windows? SWE-bench Verified uses real issues and tests, exposing retrieval, patching, and validation quality. OSWorld, WebArena, and WorkArena examine reliable interaction with environments. Terminal-Bench stresses shell-native systems. Leaderboards such as HAL also consider reliability and cost.

The competition is therefore increasingly between complete systems. When reading a score, ask not only which model produced it but which harness it ran in. Otherwise, you may mistake a system score for a model score.

More Open Source, but System Capability Remains Scarce

Claude Agent SDK, Anthropic’s multi-agent research system, SWE-agent, SWE-ReX, deepagents, Harbor, AgentKit, and Terminal-Bench offer useful implementations and references. Their surfaces differ, yet the same underlying needs recur: session and state management, controlled execution, task decomposition, recovery, verification, traces, and observability.

CLI-Anything and HiClaw, discussed in the Alibaba article, fit this pattern too. One translates GUI capabilities into CLIs agents can operate consistently. The other emphasizes multi-agent organization, shared filesystems, model routing, permissions, FinOps, and gateway controls. Different products, but both build a more complete execution environment.

Anthropic’s research system is especially instructive. Multi-agent work is useful when a large problem can be divided into parallel, compressible subproblems and combined by a lead agent. Subagents expand available token capacity, isolate context, and compress information. But they also consume many tokens and can sharply increase coordination complexity. Research tasks often parallelize well; coding tasks with tight dependencies and constant coordination may not.

That observation cuts through a misconception: multiple agents are a means; reliable handoffs, compression, and recovery are the goals.

The more open-source systems I study, the less I believe a particular framework API is the scarce ingredient. The real question is whether environment, state, rules, verification, and recovery are connected coherently.

Tencent warns against interpreting harness engineering as another collection of tools, frameworks, and platforms. Read OpenAI, Mitchell, Anthropic, HumanLayer, and Inngest together and the challenge is to transfer engineering skill into environment design. How should the system be organized so an agent understands it? How should rules be expressed so it follows them? How should errors be returned so it can repair them? How should knowledge persist so the next session avoids the same trap? That shift is the hard part.

In Enterprises, a Harness Also Governs Collective Intelligence

The Alibaba article takes the discussion beyond a coding agent’s workbench into enterprise operations.

Improving an individual agent’s efficiency matters. But organizations usually need more than faster writing before making sustained investments. They need multiple roles, tools, models, and knowledge sources organized into a governable, orchestrated system that can evolve over time.

That expands the questions:

How do agents avoid contaminating one another’s memory?
How are files and intermediate results shared?
How are tasks routed to appropriate models?
How are credentials, permissions, audits, and rate limits handled?
How are costs controlled and failures degraded gracefully?

At enterprise scale, the harness must let multiple agents work without destabilizing each other or escaping governance. This is consistent with the problems projects such as HiClaw address.

The term will become more common because the move from individual copilot to organizational executor inevitably requires this layer.

Three Conclusions I Now Hold More Clearly

First, prompt engineering is still useful, but it is one component. Its effectiveness depends on connections to tools, state, standards, verification, recovery, and evaluation. Studying prompts alone is a little like polishing the steering wheel while ignoring the car.

Second, the most valuable engineering skill may increasingly be arranging the work. OpenAI’s “Humans steer. Agents execute.” describes a role shift. Valuable engineers will decompose problems into manageable tasks, preserve experience in durable artifacts, turn taste into executable rules, expose runtime feedback, and design boundaries, recovery, and cleanup. The advantage may move from typing speed to the quality of the working environment.

Third, harness engineering gradually removes what people call the “AI feel.” Vague requirements produce empty prose. Unclear acceptance criteria produce premature confidence. Dirty context causes repetition and drift. Missing feedback creates busy activity without completion. Much of that character may come from a poor harness rather than the model alone. The practical work is to remove ambiguity, noise, and reliance on luck from the system.

How to Start Today

After all this discussion, we should return to action. Otherwise, this becomes another article that seems sensible but changes nothing.

You do not need a major laboratory’s complete system to improve your own workflow. Start with a few things.

Create a short, stable entry file. Include repository structure, build commands, output requirements, restrictions, and pointers to detail. Keep durable knowledge—standards, templates, references, and past decisions—in predictable filesystem locations instead of relying on chat history and repeated copy-and-paste.

Define acceptance criteria before asking the agent to work. “Make a page,” “write an article,” or “fix a bug” does not explain what completion means. Then expose results: run tests, capture screenshots, read logs, and replay behavior where possible. Give the agent evidence rather than asking it to edit blindly.

Finally, turn each recurring mistake into a durable constraint: lint, a checklist, a template, a script, documentation, or a hook. Do not make the next occurrence depend on your personal vigilance. For an individual, this is a step toward a reliable collaborator; for a team, it is the beginning of its first harness.

The result may still be rough, but the workflow is already more than conversation. AI begins to feel like something working inside an environment rather than intermittently producing answers.

Tencent’s discussion offers another practical step. Useful specifications prevent guessing, rework, and amplification of mistakes. When a review reveals that an agent misinterpreted a field, guessed a cross-service contract, or missed an edge case, ask whether the definition should move upstream into the spec before the next implementation.

That is the most concrete starting point: gradually turn real failures into an environment that prevents their recurrence.

Final Thoughts

I no longer see harness engineering as merely a marketing phrase.

It looks like a redistribution of software-engineering responsibilities forced by practical experience in the AI era.

Previously, experienced engineers could compensate for weak documentation, loose boundaries, messy logs, vague specifications, and empty acceptance criteria. As agents take on execution, those weaknesses are multiplied. What once looked merely unprofessional can now make an agent ineffective.

Ultimately, harness engineering asks a new version of an old question:

What kind of system is fit for sustained collaboration?

We used to ask that of people.

Now we have to ask it of agents too.