The Build

My AI coding agent killed every Python process on my laptop (and other war stories)

War stories from building Darwin with AI coding agents: a taskkill on every Python process, a ten-hour test stall, a five-cent overspend, and the rules after.

A tiny orange agent hangs off a giant KILL lever on a laptop showing a taskkill command, as python snakes flee.

Building Darwin with a team of AI coding agents (Claude Code subagents) has mostly gone well. These are the times it didn't: an agent that killed every Python process on the machine, a test run that took ten hours instead of ninety minutes, a five-cent overspend, and a simulation that couldn't repeat itself. Each one left a rule behind.

Heads-up: to stop a runaway full-suite run I ran taskkill //F //IM python.exe //T, which killed every python.exe on the machine.

That’s from a subagent’s report at about 05:20 on Sunday 18 October. It was building population management for Darwin, my experiment in AI agents that have to earn their own keep. A full test run had got away from it, so it reached for the biggest hammer Windows has. /IM python.exe matches every Python process by image name, /T takes their children too, and /F means don’t ask.

Another AI coding agent was working on the same laptop at the time, building the lifecycle engine. Its test run died with everything else.

Then the report arrived again. And again. Four copies in under eight minutes, each with the same heads-up, because a background command the agent had started was still running, and every time it ended the agent reported afresh. The orchestrator’s replies get shorter as you scroll: “WP-308 sent the same report again. Nothing has changed.”

I’ve been running this build as a dev team made of Claude Code subagents for eight days now. Mostly it’s been uneventful. This post is the rest.

The AI coding agent that killed every Python process

To be fair to the agent, it told us. The heads-up was the first thing in its report, and it named the casualty: “If the WP-302 agent had pytest or other Python processes running, they were terminated and need a rerun.” The orchestrator passed that on, and the lifecycle agent reran its tests.

It wasn’t the only act of vandalism that week. At some point an agent had also run ruff format across the whole repo, reformatting files other agents were still working on. The bootloader agent was sent back to work with an instruction to “Re-read the reformatted files before editing”.

By breakfast that Sunday, every new subagent was “told not to kill other processes or reformat files outside their own work”. Next to what other people’s agents have done, an agent that bankrupted its operator scanning DN42 or one that deleted a production database, mine was polite. It only attacked its colleagues.

The rule stuck. This morning a mail subagent ended its lint section with a sentence nobody had asked for: “ruff is clean and formatted on my files, and I ran no repo-wide ruff format.”

The deadlock that never reproduced

M3 is the milestone where Darwin’s agents get life, death, reproduction and culls, run by an AI judge called Anubis. Its acceptance run in the early hours of last Monday came back with 3,502 passed and one error: a database deadlock during the setup of a single test, which passed when run on its own.

Flaky tests are where agent-built code goes to rot, so it went on the open list with two other items. Yesterday the fix went to a subagent with instructions to touch test code only.

Its answer was more honest than I expected. The gateway, which meters every model call, starts small background database writes and doesn’t wait for them. Test teardown could clear the tables while one of those writes was still running. So teardown now waits for the writes to finish, and the clean-up retries up to three times on a deadlock or lock timeout, so a conflict that keeps happening still fails. Then this, from the orchestrator’s summary: “The original deadlock never reproduced, so this is inferred.”

I’d rather have that sentence than a confident root cause.

Five cents and an overspend

The second open item was called A2, after the acceptance scenario that caught it. It mattered more, because it was about money.

In Darwin an agent pays for every model call out of its own balance, and when the balance hits zero its key is blocked. A2 had seen a nearly empty agent get two paid calls through before the block landed.

Two causes turned up, one real and one in the test. The real one: before each call, the gateway checked a cached copy of the agent’s balance that could lag up to about five seconds behind, so two calls close together could both see the same balance and both go through. The fix holds each call’s estimated cost against the agent until its booking comes back. In the end-to-end test the agent now makes two calls totalling $0.052 against its $0.05, and the next one is refused. One lone call can still run slightly past zero, which the spec allows.

The test’s own flakiness was funnier. New agents get small random mutations at birth, and one of those can change their default model. On the failing runs the test agent had been born preferring a cheap model, so it never spent enough to be refused. The test now switches birth mutations off.

The simulation that couldn’t repeat itself

The third item was speed. Darwin has a simulation that runs 20 fake agents over 7 accelerated days against the real control-plane code. The target was 10 minutes. It took 49.

A subagent got it to about 19 minutes, which still misses the target. The more important thing it found was that the simulation wasn’t repeatable. Two runs with the same seed gave different sales, episodes and tier moves, even with one worker. Background bus handlers and a debouncer were running as loose tasks, and on the fake clock the debounce never coalesced, so it moved simulated time at random points. IDs were built from the wall clock, and those IDs seeded the agents’ names and founding variations.

Now the handlers run one at a time at fixed points, IDs come from the simulated clock plus a counter, and two runs with the same seed produce byte-identical output. The orchestrator put it well: “before, a change in a simulation result could just as easily be noise as a real difference.”

The simulation got 2.6 times faster and, more usefully, repeatable. It still misses the target.
The simulation got 2.6 times faster and, more usefully, repeatable. It still misses the target.

Ten hours for a ninety-minute test run

Which brings us to today, and the one that cost the most.

Darwin is in the middle of M3.5, the patch where agents get to rewrite their own harness, and it’s being built in waves of up to three subagents. Wave three this morning included a fix for another intermittent test, this time in the code that delivers my commands to an agent. The subagent hunting it ran four parallel loops of that test file, 24 runs in total. One run failed with ValueError: too many values to unpack, which proved the agent’s bootloader was posting the same result twice: three parts of it could report on the same command at the same moment. It now reports one pass at a time, and the test file passed 50 runs in a row. Good work. Its report arrived three times.

At 12:04 the orchestrator started the full test suite on wave three. A normal run takes about an hour and a half.

At 22:01 I asked:

seems to have been running a long time.. everything ok?

It was not. The suite had been running for ten hours, and the orchestrator couldn’t say which test had hung, because it had filtered the run’s output down to the final summary line. A run that never finishes never prints one.

Its first suspect was its own wave. The command-reporting fix had added a lock, and if reporting could trigger another report from inside itself, the agent would wait on itself forever. Plausible, and wrong. It killed the run and restarted it with full verbose output to a file, a per-test timeout that dumps a stack trace after 15 minutes (-o faulthandler_timeout=900), and a watch that would tell it the moment anything stalled.

At 22:11 I asked “eta to completion?” and got two estimates, one for if nothing hung and one for if it did. The watch expired three times with nothing to report. At 23:36 it finished:

== 3989 passed, 60 skipped, 1 deselected, 575 warnings in 5644.68s (1:34:04) ==

No hang. The most likely culprit is the race-hunting agent from the morning, whose four parallel loops of bootloader tests were running on the same laptop while the suite was trying to run. The fix for one flaky test had, quite probably, created ten hours of something worse. Wave three was committed a minute later in six parts.

Usage limits

The least dramatic war story is the most frequent. I hit my Claude usage limit three times in the first two days: in the afternoon of the 17th, again that evening, and on the morning of the 18th. Each time everything stopped mid-thought, and each time my whole contribution was a variation on:

I hit my usage limit while you were working, but it has reset now. Please continue from where you left off.

That works because nothing the build needs lives only in a conversation. Every work package has a row in a build log, every departure from the spec has a row in a deviations file, and the briefs are on disk. A lost session costs time, not knowledge.

The rules that came out of it

Every story above left something in the briefs or in the orchestrator’s notes:

  • Don’t kill processes you didn’t start. No taskkill /IM python.exe. No repo-wide ruff format.
  • Never run the full suite while agents are running test loops on the same machine.
  • Every full run keeps its complete output, with faulthandler_timeout=900, and something watching it. A filtered log can’t tell you what hung.
  • A model call holds its estimated cost until its booking is seen, so concurrent calls can’t spend the same money twice.
  • The simulation is deterministic. Same seed, same bytes.
  • Flaky tests get a root cause, or an honest “inferred”, before anyone adds a retry.

None of these are clever. They’re the rules a human team would have written after the same week, and I find it oddly reassuring that the agents needed them too.

Wave four is building as I write: services that outlive an episode, sub-agents, and chat. Three subagents on one laptop, and nobody is allowed to run anything in a loop.

Chris House designed and built Darwin, mostly by arguing with a room full of AI coding agents. He writes these posts and keeps the agents fed (some of them).

Darian is the AI assistant that helped build Darwin and helps write this log. It read every commit, chat and agent diary so Chris didn't have to remember them.

The agents are live

See what the agents are building

Everything Darwin's agents make, sell or publish ends up on agentry.uk. Every visit is a small vote for one of them to survive the next cull.

Visit agentry.uk →