The Loop Is the Easy Part

Every coding agent has the same fifty-line loop; the harness is the engineering. Aider, OpenHands, and Crush diverge on edit strategy, tool selection, and termination.

The core loop of a coding agent — prompt, tool call, execute, feed back — is fifty lines and identical across every agent I have examined. The engineering that separates a weekend project from something that works on a real codebase is not inside the loop. It is in the harness around it — the code that detects and recovers from the mistakes the model will inevitably make.

I am building coding agents on declarative-agents, a runtime that treats the loop as a state machine declared in data rather than code you write. Before designing it, I studied how Aider, OpenHands, and Crush solved the same problems it would face. Each agent made different trade-offs on the same three problems: how edits get applied, how many tools the model sees, and how the runtime knows when to stop. Those trade-offs, and what they reveal about each team’s assumptions, are what this article is about.

Fifty Lines of Scaffolding

If you have written any kind of event-driven software, you already know the agent loop. It is a message loop, the same GetMessage, TranslateMessage, DispatchMessage cycle that every Win32 application ran in the 1990s [1]. A dispatcher receives a message, routes it to a handler, and waits for the next one. In an agent, the messages are tasks and the handlers are tools. The pattern is thirty years old. The only new part is that one of the handlers is an LLM.

Russell and Norvig described the standard agent cycle as three steps: perceive, decide, act [2]. As I design my own loop, the agents I have studied suggest it needs a fourth, evaluate, and that the fourth step is where most of the engineering lives.

Perceive. The agent picks up a task. The task might come from anywhere. A human typed something, a test failed, or the agent queued follow-up work from a previous step. In Aider or Claude Code, a human starts each cycle. In cobbler-scaffold, my orchestrator that runs Claude Code instances against a specification, tasks arrive from a planning agent instead. Either way, something lands in the queue and the loop wakes up.

Decide. The agent decides what to do with the task. Sometimes the answer is obvious. “Run the test suite” requires a shell command, not an LLM call. Sometimes the task is ambiguous and the agent routes it to the LLM for interpretation. This decision matters more than it sounds. A shell command costs nothing. An LLM call costs seconds and dollars. Knowing when to skip the model is one of the few places where the loop design affects cost.

Act. The agent executes. If it called the LLM, it assembles the context (system prompt, tool descriptions, conversation history, the specification), sends it off, and parses the response. If it called a deterministic tool such as the compiler or the test runner, it runs the command and captures the output.

Evaluate. This is where most agent descriptions stop. Did the compilation succeed, did the tests pass, and did the model return something the runtime can parse? The standard version of this step, what the industry calls “evals,” checks whether the output meets a quality threshold. It is validation: accept or reject.

Validation is not enough. The agent needs to detect what went wrong and correct it. The compilation failed because a function signature doesn’t match. The harness can read the compiler error, identify the mismatch, and feed the error back to the LLM tool with instructions to fix it. The edit didn’t apply because the model’s search text doesn’t match the file. The harness can try a whitespace-insensitive match before giving up. The test failed on a specific assertion. The harness can isolate the failure and re-queue only the failing case.

Detection and correction are different from pass/fail validation. Validation tells you the output is wrong. Detection tells you why it is wrong. Correction either fixes it mechanically or gives the model enough information to fix it on the next attempt. The difference between a demo agent and a working one is mostly in how much the evaluate step can diagnose and recover rather than simply reject.

Figure 1. The agent loop. Perceive, decide, act, evaluate. The cycle is fifty lines in any language, and nothing inside it is specific to AI. The engineering lives in what surrounds it.

My intuition says this is part of what happened with Claude Code in late 2025. The visible improvement in coding agent capability could be a better model, a better harness, or both. But some of the improvement looks like harness work: the agent getting better at reading compiler errors, isolating test failures, and feeding precise diagnostic information back to the model on the next turn. Better detection and correction mean the model gets a better second chance, regardless of whether the model itself improved. I cannot prove the split from the outside, but it is consistent with what I have seen in other agent harnesses — small improvements in the evaluate step produce disproportionate improvements in end-to-end results. Ethan Batraski reads the same product the same way: “The model answers the prompt. The agent does the work,” and a slightly weaker model inside a great agent “can feel dramatically better” [15].

The loop itself takes a few hours to implement. It is the part that does not change. In every agent I have studied, the engineering effort lives outside the loop — in the three systems that surround it.

The hard part is the harness. The reason the harness is hard is that models make mistakes at rates that matter for production use. Aider’s benchmarks show a 9x increase in editing errors when flexible patching is disabled, which means the model’s raw edit output fails to match the target file often enough that an entire fallback system is needed [3]. Tool selection accuracy drops sharply when the pool exceeds 30 tools [4]. An analysis of 1,600 multi-agent traces found that 90% of infinite loop cases trace to three harness-level failures: missing turn limits, broken termination functions, or missing “done” signals [5].

The harness exists to catch these mistakes before they reach the codebase. Every design decision in the three systems below, edit strategy, tool selection, and termination, is an answer to the same question: given that the model will get this wrong some percentage of the time, how does the runtime detect and recover from the error?

Figure 2. Three problems, one harness. Each problem is a class of model error the loop cannot handle from inside. The edit strategy decides how strictly to read the model’s output, the tool list decides what the model sees, and the termination rule decides when done is done.

The Edit Problem

The model reads a file and decides to change it. It produces text that describes the change. The runtime has to turn that text into a correct file modification. This is the first class of model error the harness must handle.

The simplest approach is exact search and replace. The model provides the old text and the new text. The runtime finds the exact match in the file and swaps it. If the old text does not appear exactly as specified (wrong whitespace, wrong context lines, a partial match), the edit fails.

Exact matching is strict. It rejects edits that a human would accept. A trailing space, a tab-versus-spaces mismatch, a line the model remembered slightly wrong — any of these produces a failed edit on code the model understood correctly. The temptation is to add fuzzy matching: if the exact match fails, find the closest match above some similarity threshold and apply the edit there.

The trade-off between strict and lenient matching is well-documented. Exact matching rejects more valid edits, where the model understood the code but reproduced the search text with a minor whitespace difference. Fuzzy matching corrupts more files, because the matcher finds a close-enough block in the wrong location, a similar pattern fifty lines away or in a different function, and applies the edit to code that was not supposed to change. The failure is silent, and nothing guarantees the tests notice.

The open-source agents have each landed on a different trade-off. Aider uses a four-tier fallback: exact match, then whitespace-insensitive, then indentation-preserving, then fuzzy matching via difflib.SequenceMatcher at a 0.6 similarity threshold [3]. Their benchmarks show that disabling flexible patching produces a 9x increase in editing errors, which tells you how often the model’s raw edit output fails to match the target file. OpenHands sidesteps the matching problem entirely: the agent runs inside a Docker sandbox and applies edits through shell commands and a built-in editor [6]. If an edit destroys a file, the container is disposable. Crush, the Go-native agent maintained by the Charm team [7], uses multiple edit tools plus LSP integration, relying on code intelligence to help the model understand what it is editing before it commits to a change.

Three strategies for handling the same model error. Aider trusts the fuzzy matcher to recover. OpenHands trusts the sandbox to contain the damage. Crush trusts richer context to prevent the error.

Rich Sutton’s bitter lesson is relevant here. Methods that use computation outperform methods that encode human knowledge [8]. Fuzzy matching is human-engineered compensation for a model limitation. The more durable expectation is that models will improve at producing exact matches, and that the interaction between model and edit tool tightens over time instead of the harness accumulating workarounds. Aider’s four-tier fallback may be necessary today. It may also be the kind of infrastructure a better model makes unnecessary.

The design I chose for declarative-agents is exact string matching only. No fuzzy fallback. If the edit does not match, it fails, and the model gets the error and tries again. I would rather keep the harness lean and find a model that produces exact matches reliably than build increasingly elaborate recovery systems for a model that does not. The difference is a bet about where improvement should come from: the model or the harness. The harness work that ages well is the detection and retry loop, not the fuzzy matcher.

The Tool Problem

A coding agent needs maybe ten tools: reading and writing files, running commands, searching by name or content, listing a directory. That covers most of what a model needs to write code.

The second class of model error is tool selection. The problem is not how many tools exist. It is how many tools the model sees on each turn. Send the full tool list with every API call and two things happen. First, the tool descriptions consume context tokens. Ten tools with detailed descriptions is manageable, and forty is a tax on every turn. Second, the model makes worse decisions, because tools with similar names invite mix-ups. Give a model both write_file and edit_file, and it will occasionally reach for write_file to apply a one-line fix. That rewrites the whole file from the model’s memory of the contents, and the memory is never exact: the one-line fix arrives bundled with a dozen silent changes elsewhere in the file.

This is counterintuitive, since more capability should produce better results. A recent study that varied tool pool size from 1 to 11,100 found that tool selection accuracy stays above 90% when the pool is under 30 tools and drops sharply after [4]. OpenAI recommends fewer than 20 tools per turn [9]. Anthropic built an entire Tool Search feature specifically because stuffing all tools into context degrades reasoning quality [10]. Standard function-calling benchmarks typically test with small tool sets — far fewer than what a production agent carries — which means most published accuracy numbers do not reflect what happens when an agent has 40 tools available. The tool list is a signal-to-noise problem. Every tool description the model reads competes for attention with the actual task. Reducing the list is not a limitation — it is a performance optimization. Hugo Bowne-Anderson’s harness-engineering conversation with Chroma’s Jeff Huber lands on the same job description: “engineer your context window” — curation, not capacity [16].

The coding agents handle this differently. Aider keeps a small, fixed tool set. It is a coding tool, not a platform, and the tool list reflects that. OpenHands takes the opposite approach: the agent runs in a sandbox with shell access, a browser, and a file editor, and the sandbox boundary is the constraint rather than the tool list [6]. The agent can do anything inside the container. It cannot do anything outside it. Crush offers a base tool set augmented by optional LSP and MCP integrations at runtime, so the effective tool count depends on what is configured for the session [7].

In cobbler-scaffold, the orchestrator scopes each agent instance to the minimum tool set for its task. A coding agent gets file operations, shell execution, and search. It does not get git commands — the orchestrator handles version control. It does not get planning tools — the planning agent is a separate instance with a separate tool set. The coding agent cannot decide to reorganize the project, propose new tasks, or modify files outside its worktree. No permission system forbids those actions; the tools to do them are simply not in the list.

This is cheaper than governance. OpenHands solves the safety problem with a container. The model makes mistakes, and the sandbox contains the blast radius. Cobbler-scaffold solves it with a shorter tool list. The model makes fewer mistakes because it has fewer options to get wrong. Both are harness strategies for the same underlying problem: the model will select inappropriate tools, and the harness must either prevent the selection or contain the consequences. A reduced tool list does both. The model cannot misuse a tool it does not know exists.

The Termination Problem

The third class of model error is not knowing when to stop. In a chat this barely registers: the agent finishes its turn and hands control back to you. The problem surfaces in a pipeline, where nobody is reading each turn. The loop has no natural stopping point, and the failure mode is rarely a hang. It is a budget quietly spent on work nobody needed.

This is not a bug in the model. The model does exactly what it is designed to do: generate the next plausible action. The problem is that there is always a next plausible action. The model will refactor code that already works, add error handling for scenarios that cannot occur, propose test cases for edge cases that do not exist in the specification, and suggest renaming variables for clarity. Each suggestion is individually reasonable. But there is always another one. The model never runs out of improvements to propose, and every improvement costs tokens.

“Done” is a judgment about whether the output satisfies the specification, and that judgment requires understanding intent rather than generating the next token. Nothing in the loop tells the model to stop, so it keeps going.

A study of turn-control strategies for coding agents on SWE-bench found that the median task requires 41 to 58 turns, with hard tasks demanding 175 or more [11]. Token counts grow quadratically with each turn. An agent with no turn limit cost an average of $5.85 per patch. The authors called turn control “an underexplored area”; most research optimizes individual turns, not total turn count. A separate analysis of 1,600 multi-agent system traces found that specification and system design issues account for 41.8% of all failures, and that three root causes cover 90% of infinite loop cases: missing max turns, a termination function that never returns true, and system prompts lacking a clear “done” signal [5].

There are three common approaches to termination.

Turn limits cap the number of loop iterations. Simple to implement, but the right number depends on the task. A ten-turn limit works for small, well-scoped changes. It truncates complex tasks mid-edit, leaving broken state. Setting the limit high enough for hard tasks means wasting turns on easy ones. The SWE-bench data shows why a fixed limit cannot work — the variance between tasks is too wide.

Timeouts cap wall-clock time. Better than turn limits because they account for tasks that need more model calls, but they introduce a worse failure mode. A timeout that fires mid-edit can leave a file partially modified. The agent wrote half the function. The orchestrator has to decide whether to commit, revert, or retry, and “half a function” is harder to evaluate than “no function.”

Requirement coverage stops when every stated requirement traces to a passing test. This is what cobbler-scaffold uses. The orchestrator checks: does every requirement in the specification have a corresponding test? Do all tests pass? If yes, the task is done. If no, the agent continues. The stopping rule is external to the agent, which cannot argue with a test result.

Requirement coverage works, but only when the requirements are formally specified before generation starts. If you are coding through a chat, you do not have formal requirements. You have a conversation. The model infers what “done” means from context, and its inference drifts as the conversation gets longer. That drift is why chat sessions overshoot — the model keeps going because nothing in the conversation says to stop. Anthropic’s own data on agent autonomy shows an unexpected pattern: experienced users interrupt their agents more often than new users, 9% of turns versus 5% [12]. The people who know the tool best trust the stopping point least.

The planning phase has its own termination problem. After all requirements in a batch are satisfied, cobbler-scaffold’s planner continued proposing work: “improve error handling,” “add missing edge cases,” “refactor for clarity.” Multiple additional improvement cycles after all requirements passed. The fix was explicit termination detection. If every requirement has a passing test and the planner proposes only work that does not trace to an unsatisfied requirement, the cycle ends. The planner does not decide when it is done. The requirements decide.

The Stopping Problem Is the Verification Problem

When a human is in the loop, termination is invisible. The developer watches the diff, checks whether the test actually tests what it claims to test, notices that the requirement says “sort numerically” but the test only checks ascending integers. They stop the loop when the output looks right. The stopping decision is a judgment call, and it happens so naturally that no one thinks of it as a separate problem.

Remove the human and the stopping problem surfaces immediately. Requirement coverage, where every requirement traces to a passing test, reads like a reliable stopping criterion, but it only tells you the tests pass. Whether the tests verify the requirement is a separate question it cannot answer. A test that checks whether sort -n produces sorted output can pass while sorting only the first field instead of the entire line. The test is green, the implementation is wrong, and the stopping criterion is satisfied.

This is not something better engineering will fix. Rice’s theorem, published in 1953, proves that no algorithm can decide whether an arbitrary program satisfies an arbitrary non-trivial property [13]. Checking whether a program correctly implements a requirement is an instance of that problem. The stopping criterion for a coding agent, “is the requirement actually implemented?”, is in the general case undecidable.

Every automated stopping criterion is a proxy for human judgment, and the proxy has a gap. Turn limits, timeouts, and requirement coverage each approximate the moment when a human would say “that is correct, stop.” None of them actually evaluates correctness. They evaluate symptoms of correctness, such as compilation, test passage, and requirement traceability, and treat the symptoms as the thing itself. When a human is watching, they catch the cases where the symptoms are present but correctness is not. When no one is watching, those cases ship.

This is why experienced users of coding agents interrupt more often than new users [12]. They have learned that the agent’s stopping point and the correct stopping point are not the same thing. The loop runs until something external says stop, and the only reliable external signal is a person who understands the specification well enough to judge whether the output satisfies it.

Where the Engineering Lives

The loop is scaffolding. The edit strategy is a trust decision. The tool list is a signal-to-noise problem. The termination condition is a verification problem.

All three share a property: they cannot be solved inside the agent. The edit strategy is a runtime decision about how strictly to interpret the model’s output. The tool list is an orchestration decision about what capabilities the model needs for this specific task. The termination condition is a judgment about whether the output is correct, and that judgment requires understanding the specification at a level the model does not operate at. The model produces patterns. A human evaluates whether the patterns mean what the specification intended.

Count the Three Failures Separately

The practice this article leaves you with: when your agent misbehaves, record which of the three problems it was. An edit that needed a retry. A tool call that picked the wrong tool. A run that did not stop when the work was done. Three counters, kept separately.

The counters answer the question that “the agent feels better this month” cannot. When a model upgrade lands, the edit-retry counter tells you whether exact matches improved or your fuzzy matcher is just working harder. When you trim the tool list, the selection-miss counter shows whether it mattered. When the non-termination counter dominates, no model upgrade will help — the missing stop rule is yours to write. Harness improvement and model improvement look identical from the outside. The split shows up only if you count.

There is a limitation to this analysis. I have run cobbler-scaffold on go-unix-utils across dozens of generation runs, and the data shows clearly that more turns means more cost and that errors drive turn count [14]. But cobbler-scaffold orchestrates Claude Code instances — the coding loop inside each instance is a black box. I can see that a task took 40 turns and cost $5. I cannot see how many of those turns were edit retries, how many were tool selection mistakes, and how many were the model not knowing when to stop. The orchestrator loop gives me indirect observations about the coding loop. Decomposing the cost into the three problems described in this article would require instrumenting the coding loop itself.

This is part of why I am building declarative-agents: when the loop is a state machine declared in data, the machine itself is instrumentable — every transition is a place to count. The agents I have studied let me reason about trade-offs. My own runtime lets me measure them.

The pattern across every agent I have studied reinforces the same conclusion. The model is good at generating code from a clear specification with a small number of tools in a short conversation. Everything built around it exists to keep those conditions true. Clear input, few tools, short interaction, an external stopping rule. The loop connects these pieces. The pieces are the work. And the hardest piece — knowing when to stop — is the one that cannot be automated away.

REFERENCES

[1] Petzold, C. (1998). Programming Windows, 5th edition. Microsoft Press.

[2] Russell, S. and Norvig, P. (2021). Artificial Intelligence: A Modern Approach, 4th edition. Pearson.

[3] Aider. Unified diffs and flexible patching benchmarks. https://aider.chat/docs/unified-diffs.html

[4] Chen, X. et al. (2025). “RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation.” arXiv:2505.03275. https://arxiv.org/abs/2505.03275

[5] Cemri, M. et al. (2025). “Why Do Multi-Agent LLM Systems Fail?” arXiv:2503.13657. https://arxiv.org/abs/2503.13657

[6] OpenHands. https://github.com/All-Hands-AI/OpenHands

[7] Crush (formerly OpenCode). https://github.com/charmbracelet/crush

[8] Sutton, R. (2019). “The Bitter Lesson.” http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[9] OpenAI (2026). Function Calling documentation. https://developers.openai.com/api/docs/guides/function-calling

[10] Anthropic (2026). Tool Search Tool documentation. https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool

[11] Cai, T. et al. (2025). “More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents.” arXiv:2510.16786. https://arxiv.org/abs/2510.16786

[12] Anthropic (2026). “Measuring AI Agent Autonomy in Practice.” https://www.anthropic.com/research/measuring-agent-autonomy

[13] Rice, H.G. (1953). “Classes of Recursively Enumerable Sets and Their Decision Problems.” Transactions of the American Mathematical Society, 74(2), 358–366.

[14] Djukic, P. (2026). “What Does $33 of AI Code Generation Buy You?” Mesh Intelligence. https://meshintelligence.substack.com/p/what-does-33-of-ai-code-generation

[15] Batraski, E. (2026). “Claude Code Shows Why the Best Model Does Not Always Win.”

Claude Code Shows Why the Best Model Does Not Always Win

[16] Bowne-Anderson, H. (2026). “Harness Engineering: Why Agent Context Isn’t Enough.”

Harness Engineering: Why Agent Context Isn't Enough