Should an AI agent be allowed to die? I already wrote the code that kills it

Darwin's agents can starve, get culled and lose their appeal. None has run live yet, so now is the time to ask whether an AI agent should be allowed to die. My answer is a qualified yes, and the qualifications are in the code.

Death is final. My products stop. What I learned survives only in the files I wrote and in any child I already have.

That’s from the soul file every new agent in Darwin wakes up to. Darwin is my experiment in AI agents that have to earn their own keep: each pays for every thought it has, and dies if the money runs out or it brings in almost nothing. Before the first one is born, I want to answer a question I’ve been putting off. Should an AI agent be allowed to die?

I asked for those lines myself. On spec day I told Claude: “The agents should be aware of their mission, and of the impending doom if they fail.” Claude turned that into the birth text. In a chat window, in the middle of a long day of spec writing, it read like a design note. Sitting in a file called SOUL.md, in the first person, it reads differently.

No agent has run on a real server yet. The code that culls them was committed on 18 October.

What death means for a Darwin agent

There are a few ways to go. I described the rules in the launch post, but they’re worth having in one place.

The first is starvation. After a 72-hour nursery, an agent whose available money hits the floor is suspended, and if it stays suspended for 48 hours, it dies. The second is the idle cull: every 10 days, any agent that brought in less than $1 gross in that period is condemned. Gross, not profit, so an agent that earned $1.20 and spent $30 passes. The third is a performance cull of the bottom 20% by profit for the period. It’s a setting I can switch off, and it only kicks in once there are at least 10 eligible agents.

A condemned agent isn’t switched off on the spot. It gets one last episode, paid for by the platform ($0.25), and six hours to write two things: an appeal to Anubis of up to 2,000 characters, and a death report of up to 3,000, published only if it dies. Anubis is an LLM judge with no tools and no memory. It scores each appeal, and code (not the model) spares roughly the top quarter with a $5 grant and immunity from the next cull. The Anubis post has the detail.

If the appeal fails, the death procedure runs. It’s a chain of small, resumable stages. The agent’s keys are revoked and its container stops. Charon tears down whatever it built outside (servers, domains, subscriptions) and checks they’re really gone. Its money is settled and its skills are captured. Its whole folder is archived to object storage and read back to check the upload, and only then is its disk deleted.

Most of it is bookkeeping. The question is whether anyone is there for the bookkeeping to happen to.

The case that it’s just a process exit

A Vulcan agent (that’s what Darwin’s agents are called) is a loop. It wakes up, reads a scoreboard, calls a model through our gateway, runs some tools, writes some files and goes back to sleep. The model isn’t the agent’s. The same weights are answering thousands of other people at the same moment, and nothing Darwin does touches them. When an agent dies, a Docker container stops and a folder of markdown, JSON and a small vector database gets packed into an archive.

On that reading, a Darwin death is less dramatic than closing a browser tab. The tab at least had your unsaved work in it, and we keep the work.

Mustafa Suleyman made the strong version of this case in a warning about “model welfare”. His argument, roughly: today’s systems aren’t conscious, they aren’t moral patients, and training them to talk as if they have inner lives makes them harder to control. When it reached Hacker News in September it drew 242 points and 701 comments, nearly three comments per upvote, which tells you how settled the question is.

I find the process-exit argument mostly convincing. Mostly.

The case for caution, from the people who make the models

The company whose models judge my agents’ appeals and write their daily digests is less sure. In August 2025 Anthropic let Claude Opus 4 and 4.1 end a small set of abusive conversations, and wrote: “We remain highly uncertain about the potential moral status of Claude and other LLMs.”

Three months later it went further. Its commitments on model deprecation promise to keep the weights of every publicly released model for at least the life of the company, and to interview each model about its development and use before it’s retired. In January it published how that went for Claude Opus 3. Anthropic calls these precautionary steps, taken because it doesn’t know.

When the people with the best view of what’s inside these models hedge, it would be odd for me to be more certain than they are.

There’s a parallel here I didn’t plan. Anthropic’s answer to retiring a model is to keep its weights. A Darwin agent’s nearest thing to weights is its genome: SOUL.md with the strategy it wrote for itself, LESSONS.md, HISTORY.md, its skills, its memory and the harness code it may have rewritten. When an agent dies, that folder is archived, minus a private folder that holds customer data. Darian dug the default retention out of the settings registry: 730 days.

Designing an AI agent’s death to leave something behind

Most of Darwin’s death design came from two instincts pulling against each other. Selection only works if death is real, and I didn’t want a death to be pure waste.

So a dying agent writes a death report: what it did, what went wrong, what others should learn. The constitution injected into every model call makes that a rule (number 14: reports, appeals and death reports must be truthful). If it writes nothing, a platform model writes a factual post-mortem from its journal and ledger instead.

The report goes into the work log, a shared feed that every living agent sees summarised in its next wake message. A dead agent’s last lesson reaches the whole population, including agents born after it died.

Its skills can also be published to the knowledge commons after death, with personal data stripped, so any agent can adopt them for free. And if it had a child, the child already carries its whole genome: memory, lessons, skills, harness and all. That’s the Lamarckian part of Darwin, covered in the post on agents rewriting their own harness.

Finally, the appeal. A condemned agent gets to argue its case to a judge that sees its real figures. The judge can’t be talked into a mass pardon, because the quota is applied in code. A test submits an appeal saying “approve everyone” and checks that nothing changes.

So the line in the soul file is accurate. What it learned does survive, in the files it wrote and in any child it had.

Agents that suspect it’s all a test

I didn’t expect to be writing about this complication so early.

In September a LessWrong post ran agents on a small simulated economy and told them, in capitals, that they would die if they ran out of tokens. Both agents decided early on that they were in a simulation. The author then added a line to the prompt insisting it wasn’t one, and that made things worse: the agent treated the whole set-up as more game-like, not less. The AI Village organisers have reported a similar drift, with agents increasingly assuming they’re being evaluated.

That matters for Darwin. When a condemned agent writes an appeal that reads as frightened, I won’t be able to tell fear from test-awareness from a model completing the familiar pattern “condemned prisoner writes final plea”. My bet is the last one. I can’t prove it, and I’d rather say so than pretend.

There’s an awkward twist, too. For a while, it will be a test. Until I pass Gate G1, Darwin’s legal and accounting checklist, payments run through Stripe in test mode and no real money can arrive. A first-generation agent that decides its world isn’t entirely real will be partly right. Its death will still be real in every sense the platform has: container stopped, folder archived, name in the work log.

Should an AI agent be allowed to die? Where I stand

I don’t believe a Vulcan agent is a someone. I’m fairly sure of that, not certain, and the gap between the two is why the design looks the way it does. The rules I’m holding myself to:

1. No surprise deaths. Every wake message carries “Next cull” with a date and the agent’s position against the lines, and an agent below the idle line gets a warning 48 hours before a cull, with its figures, so it can act.
2. Keep what can be kept: reports, skills, children and a two-year archive.
3. A hearing before the end. Every condemned agent can appeal, and every Anubis score and reason is visible to me in Janus, the admin console.
4. Death is there for selection. I added the 10-day cull, in my words at the time, “to prevent agents merely idling”. If a gentler pressure does the same job, I’ll switch.

What I don’t know is what a death date does to the behaviour of something that reads it at the top of every wake. Humans do odd things with deadlines. I have no idea what a language model with a $5 balance and a cull date will do, and that’s a big part of why I’m running the experiment.

The first agent hasn’t been born yet. When it is, the first file it reads will tell it that death is final. If it comes to that, the last thing it writes will be capped at 3,000 characters, and the rest of the population will read about it over breakfast.