article-text.txt

"To view keyboard shortcuts, press question mark\nView keyboard shortcuts\nHow to Build a Conductor: Orchestrating Multi-Agent Loops from Scratch\nleanxbt\n@leanxbt\n·\nJul 13\n·\nFollow\n11\n59\n95K\nYou already know the single-agent loop: a while-loop, check first, agent fixes, state on disk. But a single agent hits a ceiling of task complexity, the limit of what one context can close. When the goal is large - a migration, a feature spanning half the system, parsing a big repo - it has to be broken into parts, handed to several agents, and merged back. Whoever does that is the orchestrator, the conductor. This article is a step-by-step build of a conductor, from an empty file to a working cycle.\nThe build order is strict, and it is not arbitrary: decomposition, dispatch, integration, verification, brakes. Each stage rests on the previous one, and skipping any of them is exactly the spot where a multi-agent loop later blows up. We build one at a time.\nWhat a conductor is and what it is not\nFirst, clear up a common confusion. The conductor is not an agent that does the work better than the others. It does not do the domain work at all: it does not write code, it does not edit text. Its product is decisions: how to split the goal, who gets what, in what order, when to merge, and when to stop. It is a coordinator, not a worker.\nFrom this follows something practical: the conductor should be as dumb as possible in the domain and as disciplined as possible in coordination. The more it reaches to do the work itself, the worse it conducts, because it loses the overview behind the details of one subtask. A good conductor holds the goal and the map in its head and keeps its hands off the code. This is the first design decision: separate the coordinator role from the worker role physically, with different prompts and, ideally, different model calls.\nStage 1: Decomposition - how the conductor cuts the goal into subtasks\nIt all starts with the conductor turning one big goal into a list of subtasks. This is the most underrated stage: bad decomposition guarantees the failure of everything below, and no brakes will save it later.\nA good subtask has three properties. It is as independent as possible: it can be closed without waiting on three others. It has its own check: on completion there is a command that says yes or no. And it fits one worker context: if a subtask needs decomposition itself, cut further.\npython\n#!/usr/bin/env python3\n# decomposition: the conductor turns the goal into a list of checkable subtasks\nDECOMPOSE_PROMPT = \"\"\"You are a planner. Break the goal into subtasks.\n\nGOAL: {goal}\n\nRules for each subtask:\n- It must be independently completable where possible.\n- It MUST have a concrete check command that returns pass/fail.\n- It must fit one agent context. If it does not, split further.\n\nOutput JSON list. Each item:\n{{\"id\": \"short-slug\", \"desc\": \"...\", \"check\": \"shell command\",\n  \"depends_on\": [\"other-id\", ...]}}\nOutput ONLY the JSON.\"\"\"\n\ndef decompose(goal, call_model):\n    raw = call_model(DECOMPOSE_PROMPT.format(goal=goal), temperature=0.2)\n    plan = json.loads(raw)\n    # plan validation: a subtask with no check is not a subtask\n    for t in plan:\n        assert t.get(\"check\"), f\"subtask {t['id']} has no check, reject plan\"\n    return plan\nNotice the assert: a subtask with no check command is rejected on the spot. This is not nitpicking. A subtask that cannot be checked is a subtask whose doneness the conductor will later judge by eye, and an agent's eye is too kind to itself. The check field at the decomposition stage is the foundation under the whole downstream cycle, so its absence fails the entire plan.\nThe depends_on field defines the dependency graph: what depends on what. From it the conductor understands what can be dispatched in parallel and what must wait. Without the graph it either serializes everything, losing speed, or dispatches in parallel things that depend on each other, catching races.\nStage 2: Dispatch - how the conductor picks what to run now\nWith a plan and its dependencies, at each step the conductor picks the next subtask: one whose dependencies are already closed. This is simple but important logic, because it is exactly what separates a conductor from a script running tasks in order.\npython\ndef ready_subtasks(plan, done):\n    # ready = all its dependencies closed and it is not closed yet\n    return [t for t in plan\n            if t[\"id\"] not in done\n            and all(dep in done for dep in t.get(\"depends_on\", []))]\n\ndef pick_next(plan, done):\n    ready = ready_subtasks(plan, done)\n    if not ready:\n        return None\n    # heuristic: first the one that unblocks the most others\n    def unblocks(t):\n        return sum(1 for other in plan if t[\"id\"] in other.get(\"depends_on\", []))\n    return max(ready, key=unblocks)\nThe unblocks heuristic is a small but useful detail: from the ready subtasks the conductor takes the one that unblocks the most subsequent ones. This keeps the flow of work wide and stops the conductor from getting stuck at a bottleneck by closing dead-end branches first. For a first version this is enough, smart critical-path planners can be added later.\nParallelism is decided here too. If several ready subtasks exist and are independent, the conductor can run them at once. But start with sequential dispatch: parallel workers multiply the difficulty of integration and debugging, and the speed gain only makes sense once the sequential version is already reliable. First a working conductor one at a time, then a parallel one.\nStage 3: Integration - how the conductor merges results back\nThe workers closed their subtasks, each returned its piece. Now the conductor has to merge them into a single goal state. This is the second most common source of breakage after decomposition, because pieces from different agents are not obliged to fit together, even if each is individually correct.\nThe key to integration is not to hold results in the conductor's head but to materialize them into shared state on disk that the conductor reads and updates. The worker does not tell the conductor what it did - it leaves a verifiable trace, and the conductor reads that trace.\npython\ndef integrate(subtask, state):\n    # integration goes through fact, not through the worker's report\n    # the worker already changed files / created an artifact, the conductor records it\n    state[\"done\"].append(subtask[\"id\"])\n    state[\"last_progress_step\"] = state[\"step\"]\n\n    # after each merge - a check that the pieces are compatible with each other\n    # it is not enough that each subtask is green, they must not conflict\n    r = subprocess.run(\"npm run test:integration\", shell=True,\n                       capture_output=True)\n    if r.returncode != 0:\n        # the pieces are correct individually but break together\n        state[\"conflicts\"].append({\"after\": subtask[\"id\"],\n                                   \"detail\": \"integration red\"})\n        return False\n    save_state(state)\n    return True\nThe point of test:integration separate from the subtask checks: a subtask checks its own piece, the integration test checks the seam. The classic multi-agent error is that each part is green but together it does not work, because two agents understood the shared interface differently. An integration check after the merge catches this immediately, not at the end, when untangling is too late.\nStage 4: Verification - the conductor does not believe, it checks\nNow the central principle on which the honesty of the whole construction rests. The conductor never accepts a subtask on the basis of the worker reporting \"done.\" It runs that subtask's check itself and looks at the exit code.\npython\ndef verify(subtask):\n    # the conductor runs the subtask's oracle ITSELF, the worker's report does not count\n    r = subprocess.run(subtask[\"check\"], shell=True, capture_output=True)\n    return r.returncode == 0\nThe reason is simple and universal for loops: the agent that did the work is a bad judge of its own work, it systematically overrates its doneness. The worker's report of \"done\" is not a fact, it is the opinion of an interested party. The conductor turns that opinion into fact the only way possible: it runs an independent check that the worker did not write. As long as check is set at the decomposition stage and does not depend on who performed the subtask, the conductor cannot be fooled from below.\nAnd the final oracle is on the whole goal. When all subtasks are closed and verified, that is not yet success. Success is when the end-to-end acceptance test of the goal passes, one that cannot be satisfied by closing subtasks individually.\npython\ndef goal_met():\n    r = subprocess.run(\"npm run test:acceptance\", shell=True,\n                       capture_output=True)\n    return r.returncode == 0\nThe end-to-end test also protects against a subtle breakage: the conductor, optimizing the number of closed subtasks, can split the goal into trivial dummies and briskly close them. While the end-to-end test is red, all closed subtasks mean nothing, and this keeps the conductor honest - its success is measured by the goal, not by the number of moves.\nAssembling the whole cycle\nNow the five stages fold into one working conductor cycle.\npython\n#!/usr/bin/env python3\n# orchestrator.py - the full conductor cycle\nimport json, subprocess\n\nMAX_STEPS = 12\n\ndef orchestrate(goal, call_model, run_agent):\n    plan = decompose(goal, call_model)        # stage 1\n    state = {\"goal\": goal, \"step\": 0, \"done\": [], \"conflicts\": [],\n             \"last_progress_step\": 0}\n\n    while state[\"step\"] < MAX_STEPS:\n        state[\"step\"] += 1\n        subtask = pick_next(plan, state[\"done\"])   # stage 2\n\n        if subtask is None:\n            # nothing to dispatch: either all done, or all blocked\n            if all(t[\"id\"] in state[\"done\"] for t in plan):\n                if goal_met():                       # stage 4, final oracle\n                    print(f\"Goal reached in {state['step']} steps.\"); return True\n                print(\"Subtasks closed, end-to-end red. Replanning.\")\n                plan = decompose(goal, call_model)   # the plan was wrong\n                continue\n            print(\"Deadlock: subtasks left, nothing to dispatch.\"); return False\n\n        run_agent(subtask)                          # the worker works\n\n        if verify(subtask):                         # stage 4, subtask check\n            integrate(subtask, state)               # stage 3, merge + integration\n            print(f\"  {subtask['id']} verified and merged\")\n        else:\n            print(f\"  {subtask['id']} failed the check, not accepted\")\n\n    print(f\"Cap of {MAX_STEPS} steps reached.\"); return False\nRead how the stages join. Decomposition once at the start, and again if the end-to-end test is red while all subtasks are closed - that means the plan was wrong and must be rebuilt. Dispatch picks the next ready subtask. Verification stands between the worker's work and integration: if it did not pass, we do not merge. The final oracle separates \"all subtasks closed\" from \"goal reached.\" The conductor's skeleton is all here, only the brakes remain.\nStage 5: Brakes a single loop does not have\nThe base brakes remain: a step cap, a budget, a heartbeat. But the multi-agent star has two failures of its own that need dedicated fuses, because the old ones do not see them.\nThe first is a mutual block. A worker hangs, the conductor waits for it forever, the step counter may not even grow, because formally \"work\" is going on. This is caught by a per-subtask timeout: not closed in the allotted time, the conductor takes it away and marks it stuck.\npython\nSUBTASK_TIMEOUT = 180   # seconds per subtask\nimport time\nstart = time.time()\nrun_agent(subtask)\nif time.time() - start > SUBTASK_TIMEOUT:\n    print(f\"  {subtask['id']} hung, taking it back\")\n    state.setdefault(\"stuck\", []).append(subtask[\"id\"])\n    continue\nThe second is an argument with no convergence. Two workers, or the conductor and a worker, run a question in circles, there is no consensus, and the budget melts. An ordinary circuit breaker on a repeated command does not see it, because the wordings are slightly different each time. This is caught by a round counter per fork: after N exchanges the question goes up, to a human, instead of spinning further.\npython\nMAX_ROUNDS = 3\nif state.get(\"rounds\", {}).get(subtask[\"id\"], 0) >= MAX_ROUNDS:\n    print(f\"  {subtask['id']}: argument not converging, escalating to a human\")\n    state.setdefault(\"escalated\", []).append(subtask[\"id\"])\n    continue\nThe rule is the same as in the base loop, but applied to the conductor: define the orchestrator by what it can spoil. It spoils the most when coordination runs away unnoticed, so the timeout and the round cap go in before you step away from the machine, not after the first burned run.\nState with attribution: without which you cannot debug\nOne thing worth building in right away, though the temptation to defer is strong: the state must remember who did what and on what basis the conductor accepted it. In a single loop the author is always one, in a multi-agent one, without attribution, debugging a failed run is hopeless - you see the outcome but not whose move spoiled it.\njson\n// .orchestration_state.json\n{\n  \"goal\": \"billing-webhook migration\",\n  \"step\": 7,\n  \"done\": [\"schema-v2\", \"handler-refactor\"],\n  \"decisions\": [\n    {\"step\": 5, \"by\": \"coder\", \"subtask\": \"schema-v2\",\n     \"evidence\": \"npm test schema -> exit 0\"},\n    {\"step\": 6, \"by\": \"orchestrator\", \"action\": \"replanned\",\n     \"evidence\": \"acceptance red with all subtasks closed\"}\n  ],\n  \"budget_spent_usd\": 6.40,\n  \"budget_cap_usd\": 15.0\n}\nThe evidence field on each decision is what turns debugging from guessing into reading. A conductor decision with no evidence means it acted on faith rather than an oracle, and those lines are the first under suspicion when the loop went wrong. Attribution is cheap at build time and priceless at three a.m. over a burned run.\nThe build order that works\nLet us fold it all into one route, because the order here matters more than the tools.\nFirst decomposition, yielding subtasks each with a mandatory check. Then dispatch by the dependency graph, sequential for now. Then integration through shared state on disk plus an integration test on the seams. Then verification: the conductor runs each subtask's oracle itself, and measures goal success by the end-to-end acceptance test. And only then the brakes: per-subtask timeout, round cap, budget, attribution in the state.\nJumping stages - building dispatch before decomposition yields checkable subtasks, or parallelizing workers before integration is reliable - is exactly how a multi-agent loop blows up while you sleep. As with the single loop: prove it on the sequential, one-agent-at-a-time version, harden it, then expand.\nAnd honestly about scale. The loud stories about a fleet of a hundred agents are real, but such a fleet costs more than a million a month and works because a lab pays for it. Your first conductor is three or four workers under one coordinator on a boring but large task: a migration, a refactor across half the repo, parsing a big batch of tickets. Start with a conductor you trust on one such task, where every subtask is checkable and you still read the merge with your eyes. Build that one.\nWant to publish your own Article?\nUpgrade to Premium\nleanxbt\n@leanxbt\nFollow\ni don't feel shame, i'm not capable of it"