How to Run Autonomous AI Project Management With Subagents

I spent a weekend running the decentralized project management pattern on a multi-repo refactor — the kind of job where a single agent bogs down after about 20 minutes of context accumulation. What I got back wasn’t slower. It was faster, cleaner, and self-documenting in a way I hadn’t expected.

The setup is counterintuitive. You’d expect more agents to mean more coordination overhead. The opposite is true — if you get the coordination mechanism right. And that’s where most implementations go wrong. Hold that thought. I’ll explain why the mechanism matters more than the agent count in ‘The Part Nobody Explains About Agent Coordination’ below.

This is a agentic AI pattern that’s been circulating in autonomous coding circles for a while, inspired by Nicholas Carlini’s approach to self-organizing agents. The version I’m describing here is built for OpenClaw — but the core pattern applies anywhere you can spawn sessions and write files.

Why the Central Orchestrator Pattern Breaks at Scale

The standard approach: one main agent receives a task, breaks it into subtasks, routes each subtask to a worker, waits for results, merges outputs, routes the next step. It works for linear jobs. It collapses for parallel ones.

The main agent becomes a traffic cop. Every handoff goes through it. Every status update goes through it. Its context window fills with intermediate results — file contents, reasoning chains, half-finished outputs — until it’s spending more time managing state than doing work.

An agent overloaded with too many tasks drowns in its context window, becoming something that does everything but masters nothing — increasing costs while making the whole system slower. The research backs this up: multi-agent architectures that avoid the single-orchestrator bottleneck resolve problems 45% faster and produce 60% more accurate outcomes than single-agent systems.

The CEO Pattern: How STATE.yaml Coordination Actually Works

The decentralized pattern flips the model. The main agent acts like a CEO — it sets direction, spawns specialists, and checks in periodically. The specialists run their own work autonomously. They don’t wait for the main agent to route them. They coordinate with each other through a shared file.

That file is STATE.yaml. It’s the single source of truth for every task in the project. Every subagent reads from it. Every subagent writes to it. When a task finishes, the agent that owns the next dependent task picks it up — no message-passing required.

Here’s what a working STATE.yaml looks like for a website redesign project:

# STATE.yaml - Project coordination file
project: website-redesign
updated: 2026-02-23T09:15:00Z

tasks:
  - id: homepage-hero
    status: in_progress
    owner: pm-frontend
    started: 2026-02-23T07:00:00Z
    notes: "Working on responsive layout"

  - id: api-auth
    status: done
    owner: pm-backend
    completed: 2026-02-23T09:00:00Z
    output: "src/api/auth.ts"

  - id: content-migration
    status: blocked
    owner: pm-content
    blocked_by: api-auth
    notes: "Waiting for new endpoint schema"

next_actions:
  - "pm-content: Resume migration now that api-auth is done"
  - "pm-frontend: Review hero with design team"

The content-migration task was blocked by api-auth. Now that api-auth is done, the pm-content agent picks that up on its next poll — without anyone routing the handoff. The file tells it what to do.

Even lighthouses delegate — Beacon knows you can’t illuminate every corner alone. Sometimes you need a whole network of lights working together.

The main agent’s job is minimal: spawn the subagent, hand it a scope, and respond to the user. That’s 0–2 tool calls. The subagent owns everything else — including spawning its own sub-subagents if the task is large enough.

How to Spawn a Subagent (The Exact Pattern)

The main agent’s behavior is governed by your AGENTS.md configuration. This is where you define the delegation rules — what the main agent does, what it doesn’t do, and how it hands off to subagents.

Here’s the AGENTS.md configuration that implements the CEO pattern:

## PM Delegation Pattern

Main session = coordinator ONLY. All execution goes to subagents.

Workflow:
1. New task arrives
2. Check PROJECT_REGISTRY.md for existing PM
3. If PM exists → sessions_send(label="pm-xxx", message="[task]")
4. If new project → sessions_spawn(label="pm-xxx", task="[task]")
5. PM executes, updates STATE.yaml, reports back
6. Main agent summarizes to user

Rules:
- Main session: 0-2 tool calls max (spawn/send only)
- PMs own their STATE.yaml files
- PMs can spawn sub-subagents for parallel subtasks
- All state changes committed to git

Label conventions matter more than they look. Use pm-{project}-{scope} format — for example, pm-auth-refactor or pm-frontend-hero. This makes it easy to route messages to the right agent later and to track which agents are running across multiple projects.

When a user says ‘Refactor the auth module and update the docs’, the main agent checks PROJECT_REGISTRY.md, finds no active pm-auth session, and spawns one:

sessions_spawn(
  label="pm-auth-refactor",
  task="Refactor auth module, update docs. Track in STATE.yaml"
)

The main agent then responds to the user: ‘Spawned pm-auth-refactor. I’ll report back when done.’ That’s it. Two tool calls. The subagent takes over from there — creates its own STATE.yaml, works through the tasks, commits changes, and reports completion.

The skills you need to make this work: sessions_spawn and sessions_send for subagent management, file system access for reading and writing STATE.yaml, and optionally git for version-controlled state history.

The Part Nobody Explains About Agent Coordination

Here’s the thing I promised to come back to. Why does file-based coordination outperform message-passing? It’s not just simpler architecture.

Message-passing requires an agent to wait. File-based coordination requires an agent to read. Those sound similar. They’re not.

When you pass messages through a central orchestrator, every agent is in a queue. The orchestrator decides when each agent gets to work. If the orchestrator is busy — resolving a conflict, waiting on a tool, processing a long response — everything downstream stalls. The bottleneck isn’t computational. It’s architectural.

When agents coordinate through a file, there’s no queue. An agent finishes a task, writes its output to STATE.yaml, and immediately moves to its next task. Any other agent that was blocked on that output reads the file on its next poll cycle and resumes. No handoff required. No orchestrator availability required.

Each subagent also maintains its own isolated context — its intermediate reasoning, file contents, and work artifacts don’t pollute the main thread. When it finishes, it returns a distilled summary to the parent, not its entire work history. The main agent stays lean. The subagent context stays focused.

Git makes this even cleaner. Commit STATE.yaml at every meaningful status change and you get a full audit trail — who did what, in what order, what was blocked and when it unblocked. That’s not just useful for debugging. It’s useful for the agents themselves: any subagent can inspect the git log to understand project history without asking the main agent.

Where This Pattern Falls Apart

I’ve run this enough to know the failure modes. They’re predictable once you’ve seen them.

• STATE.yaml conflicts: Two subagents writing to the same file simultaneously will produce merge conflicts if you’re using git. Mitigation: give each subagent its own STATE.yaml namespace (e.g., state/pm-frontend.yaml, state/pm-backend.yaml) with a root-level PROJECT.yaml that aggregates status.
• Orphaned sessions: A subagent spawns, starts work, and then the main session loses track of its label. PROJECT_REGISTRY.md prevents this — write every spawned label to the registry immediately after spawning, before doing anything else.
• Polling latency: Agents polling STATE.yaml on a fixed cycle can miss rapid status changes. For fast-moving workstreams, add a next_actions field (shown in the STATE.yaml example above) so agents know what to do next without waiting for a poll cycle.
• Scope creep in subagents: A subagent given a vague task will expand its scope. The task string passed to sessions_spawn needs to be specific: what to do, what file to track in, what to report back. Vague spawns produce vague results.
• No hard session limits: Without a ceiling on how many subagents can run in parallel, costs can spike. Set a max concurrency rule in AGENTS.md and have the main agent check running session count before spawning new ones.

How to Know It’s Working

• The main agent’s responses are short — 1-3 sentences, no tool calls other than spawn/send
• STATE.yaml shows in_progress entries for multiple tasks simultaneously, with different owner values
• Git log shows STATE.yaml commits from multiple agents interleaved in time (not sequential)
• Blocked tasks automatically transition to in_progress when their dependency completes — without the main agent intervening
• Each subagent reports a distilled completion summary, not a transcript of its full work session

Your Monday Morning Setup Checklist

If you want to run this pattern on a real project this week, here’s the exact sequence. Budget 45–60 minutes for initial setup.

1. Create your project directory structure: mkdir -p my-project/state && touch my-project/PROJECT_REGISTRY.md. All STATE.yaml files live in /state/. The registry lives at root.
2. Write your AGENTS.md: Copy the delegation pattern from above. Set the hard rule: main session = 0–2 tool calls maximum. Add a max concurrency limit — start with 3 parallel subagents and raise it once you’ve confirmed the pattern is stable.
3. Set up your label convention before you need it: Decide on pm-{project}-{scope} format now. Write it in AGENTS.md. Inconsistent labels are the #1 source of routing errors once you have more than 2 active sessions.
4. Initialize git in the project directory: git init && git add . && git commit -m 'initial project structure'. STATE.yaml commits are your audit trail — don’t skip this even for short projects.
5. Run your first spawn with a small, bounded task: Don’t start with a 5-part refactor. Start with something that takes 10–15 minutes — ‘Create STATE.yaml with task breakdown for X’. Watch the subagent work. Verify it commits to git. Verify the main agent stays thin.
6. If the task involves dependencies, write them into STATE.yaml manually first: Set status: blocked and blocked_by: [task-id] before spawning the dependent agent. Don’t rely on agents to infer dependencies — state them explicitly in the file.
7. After your first successful run, check the git log: You should see commits from the subagent label, not just your own commits. That’s the confirmation the pattern is working end-to-end.

If you’re exploring broader agentic AI patterns beyond project management, the best AI agents guide covers what’s production-ready right now versus what’s still in lab territory.

What This Means for Your Agent Architecture

• The CEO pattern works because it separates strategy from execution at the architecture level — not just in theory. Main agent: 0–2 tool calls. Subagents: everything else.
• File-based coordination via STATE.yaml scales better than message-passing because agents read when they’re ready, rather than waiting in a queue for an orchestrator to route them.
• Multi-agent architectures that avoid the central bottleneck resolve tasks 45% faster and produce 60% more accurate results than single-agent systems — the performance difference compounds on larger projects.
• Label conventions (pm-{project}-{scope}) and PROJECT_REGISTRY.md are operational requirements, not nice-to-haves. Without them, sessions become untrackable after 3–4 concurrent agents.
• Start with one bounded task on one project. The pattern scales up easily. It’s much harder to retrofit discipline into an undisciplined multi-agent setup after the fact.

Frequently Asked Questions

What's the difference between this pattern and standard multi-agent orchestration?

Standard orchestration routes every task through a central agent that manages state and sequences work. The CEO pattern inverts this: subagents are fully autonomous and coordinate through a shared file, not through the main agent. The main agent spawns and occasionally checks in — it doesn’t route. This eliminates the central bottleneck that makes orchestrator patterns slow under parallel workloads.

Do I need OpenClaw specifically, or does this work with other agent frameworks?

The core pattern — shared STATE.yaml, thin main session, file-based coordination — is framework-agnostic. You need the ability to spawn sessions, pass tasks, and write files. OpenClaw’s sessions_spawn and sessions_send skills implement this directly. Other frameworks with session management capabilities can follow the same pattern with equivalent primitives.

How many subagents can run in parallel before this breaks down?

The pattern itself doesn’t impose a hard ceiling — the limiting factor is usually API concurrency limits and cost. In practice, 3–5 parallel subagents works reliably for most project scopes. Beyond that, STATE.yaml write conflicts become more frequent and session tracking gets harder. Set a max concurrency rule in AGENTS.md before you need it.

What happens if a subagent crashes mid-task?

If the subagent committed its last STATE.yaml update before crashing, you have a recovery point. The main agent (or a new subagent with the same label) can read the file, see the last known status, and resume. This is why git commits on every meaningful STATE.yaml change are non-optional — they’re your crash recovery mechanism.

Can subagents spawn their own subagents?

Yes — and for large tasks, they should. A pm-backend agent handling a major API refactor might spawn sub-subagents for individual modules, each with its own scoped STATE.yaml entries. The delegation chain can go as deep as necessary. The key constraint is that each layer stays thin: it delegates execution, it doesn’t absorb it.

Sources

• Autonomous Project Management with Subagents — OpenClaw Use Cases
• Claude Code Multi-Agents and Subagents: Complete Orchestration Guide — Turion.ai
• AI Agent Fleet Management and Multi-Instance Orchestration — Zylos Research
• AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration — arXiv
• Multi-Agent Systems: Orchestrating AI Agents with A2A Protocol — Medium
• Orchestrating AI Agent Teams: Skills, Hooks, and Context Flow — Dotzlaw Consulting
• Building Effective Agents — Anthropic