In his July 13 fragments, Martin Fowler shared notes from a Thoughtworks retreat where engineers compared notes on running large language models (LLMs) in production. The topic that dominated the room was harness engineering — the practice of managing what an AI model sees, remembers, and is allowed to do, separate from the model itself.
This article unpacks what harness engineering means in practice, why self-hosted models came up as an alternative to frontier APIs, and why the retreat couldn't agree on whether any of this will still matter once models improve.
What "harness engineering" actually covers
A harness is everything wrapped around a model that isn't the model: the system prompt, the tools it can call, the context fed into each request, and the guardrails that catch bad outputs before they reach a user. Harness engineering is the discipline of designing and tuning that wrapper.
Think of the model as an employee and the harness as the onboarding packet, the tool access list, and the manager checking their work. A brilliant employee with no context on the company, no access to the right systems, and no review process will still produce inconsistent results. The retreat's engineers were, in effect, comparing notes on how to write better onboarding packets.
The specific challenge that came up repeatedly: deciding what unit of work to hand an agent. Too small a task and the overhead of context-setting outweighs the value delivered. Too large a task and the model loses track of unstated assumptions, edge cases, or security requirements that a human would have caught by asking a clarifying question.
Managing by objective, not by method
One thread from the retreat concerned how much you should specify. Traditional software engineering favors precise instructions: do exactly this, in this order. Managing an LLM agent effectively often means the opposite — stating the objective and letting the harness (plus the model's own reasoning) figure out the steps.
This is closer to how a manager delegates to a skilled contractor than how a developer writes a function. The tradeoff is control for coverage: the model can adapt to details a rigid instruction set would miss, but it can also drift into approaches the requester didn't anticipate — the "unstated assumptions" problem Fowler's notes call out.
The challenge isn't writing better prompts. It's building a system that catches the gap between what you meant and what the model assumed.
Self-hosted models as a harness decision, not just a cost one
The retreat also debated self-hosted, open-weight models as an alternative to calling a frontier model's API. The reasons weren't purely about price. Three distinct motivations came up:
| Motivation | What it means in practice |
|---|---|
| Cost control | Predictable compute spend instead of per-token API billing |
| Sovereignty | Keeping model weights and inference inside an organization's own infrastructure |
| Data security | Sensitive inputs never leave a controlled environment |
Running your own model changes the harness problem rather than removing it. You gain control over what the model sees and how it's deployed, but you also take on the operational burden of hosting, scaling, and keeping the model current — work a frontier API absorbs for you.
An open question: does this discipline survive better models?
The most candid part of Fowler's notes is the uncertainty at the end: participants disagreed on whether harness engineering is a durable discipline or a temporary workaround for models that aren't yet reliable enough to trust with loosely specified objectives.
The case for "temporary": as models get better at inferring intent and catching their own mistakes, some of today's harness scaffolding — retry logic, explicit constraint lists, narrow task scoping — becomes redundant. The case for "durable": even a perfect model still needs some interface for stating what you want, what it can access, and how its output gets checked before it affects something real. That interface is the harness, regardless of how good the model inside it gets.
What this means for builders
If you're building with LLM agents today, the retreat's takeaways translate into concrete defaults. Size tasks so a human can verify the output without redoing the work themselves. Decide upfront whether you're managing by method (precise steps) or objective (stated goal, model figures out steps) — mixing the two inconsistently is where unstated assumptions creep in. And treat the choice between a frontier API and a self-hosted model as a harness design decision, not just a line item on a budget.
Conclusion
Harness engineering doesn't have a settled definition yet, and the practitioners closest to the problem — a room of Thoughtworks engineers — left their own retreat without agreeing on whether it's a discipline that will still exist in a few years. What's clear is that the gap between "the model can do this" and "the model reliably does this correctly, unsupervised" is currently being closed by careful context and tool management, not by the model alone. Anyone deploying agents in production is doing harness engineering whether they name it that or not.