Bishop: Building an AI Agent Harness for Real Development Work

have written before about how I work with AI, and about why I break work down into small pieces instead of handing over a whole project and hoping for the best. This article is the other half of that. It is about the thing I actually built to make that way of working hold up.

It is called Bishop. It’s a harness, it’s on GitHub, and you are welcome to pull it apart.

The argument behind the whole thing is simple enough. A single agent with a good prompt does not survive real work. Roles, hard limits and a review path do.

What Bishop Is, and What It Isn’t

It took me a while to work out what to call this thing. I asked a lot of questions about it, mostly to myself, and I kept landing on what it wasn’t.

It isn’t a prompt. It started as prompts, and prompts are where most people start, but a prompt lives and dies inside one conversation.

It isn’t a framework. It doesn’t ask you to structure your project a particular way, and it has no opinion about what you’re building.

It isn’t a product. There’s nothing to buy, no roadmap and no support. If it breaks on your machine, you own it.

What it is, is a harness. A structure that sits around the model and decides what it is allowed to do, in what order, and who checks the work before it counts as done.

I built it because nothing else fitted. I had a fairly specific way of working already, and I was getting good results out of prompts, but I was surrounded by people running agents. So I did the reading. I downloaded things, I looked at things, I tried things. All of it was interesting and none of it was what I wanted, mostly because it was built around somebody else’s idea of how development should go.

So I started from the ground up. That has taken a few months to get to something I am willing to show people.

It has now been through well over 500 real tasks. Not demos, actual client and product work. That’s the number that made me comfortable releasing it, because 500 tasks is enough to find the places where a nice idea falls over.

One more thing it isn’t: autonomous. It runs unattended, it doesn’t run unsupervised. I’ll come back to that at the end.

Why It’s Called Bishop

I’m a big Alien fan. The franchise is probably my favourite of all time, and Aliens is the one I keep going back to.

Bishop is the synthetic in that film. What I liked about him, and the reason the name stuck, is that he is calm. He is measured. He would rather be truthful than quick, and he tells you plainly when something is a bad idea. The other options were Ash from the first film and David from Prometheus, and both of those are exactly the sort of synthetic you don’t want quietly making decisions in your codebase. Of all the androids in the franchise, Bishop is the one you would actually trust.

That sounds like a joke, and the naming is a bit of fun, but the personality itself is a design decision.

A model’s default behaviour is to be fast and agreeable. It wants to produce something, it wants you to be happy with it, and it will keep going long after the sensible thing would be to stop and ask. Everything about the way these tools are marketed pushes in the same direction, which is speed.

I wanted the opposite. The character brief for Bishop is calm, clinical, unhurried and dry. The point of it is to slow the agent down rather than speed it up. It’s the same instinct as asking for four options before anything gets built: put a pause in the place where a mistake is cheapest.

Once I had gone that far I went further and named the whole crew out of the franchise, because it’s a lot easier to think about Hicks handing something to Apone than about agent_02 handing something to agent_03. Four of them are from Aliens. Lambert is from the first film. I have no connection to the people who own the characters, it’s just my harness and I wanted it that way.

The Five Agents

There are five roles. In the public repo they carry the character names; in my own working setup they have plain role names, which I’ll come back to.

Bishop, the orchestrator. Reads the job, works out what needs doing, writes the plan, and delegates. That’s all it does. It’s the only agent I talk to directly.

Hicks, the junior developer. Does the work. Every piece of implementation goes through this agent, and it’s the only entry to the development path.

Apone, the code reviewer. Checks the work for security problems, best practice and documentation. Critical issues block and go back for a fix. Warnings are recorded and passed.

Vasquez, the senior developer. Escalation only. Arrives when the junior and the reviewer have failed to resolve the same critical issue between them, and not otherwise.

Lambert, the documentation writer. This one confuses people, because it isn’t writing documentation for the code. It’s the agent that writes to the state and memory system: the context and progress documents, the logs, the report at the end. The junior developer writes code and doesn’t touch documentation. Lambert writes documentation and doesn’t touch code.

Five is not a magic number. It’s just the smallest set I could get to that still separates deciding, doing, checking and recording.

The Hard Rules

The roles are only half of it. The rules are what make the roles mean anything, and the important thing about them is that they are not instructions. They’re permissions.

If you tell an agent in its prompt that it shouldn’t write code, it will mostly not write code, and then one day it will decide the sensible shortcut is to write a bit of code. If you take the ability away in the settings file, that conversation doesn’t happen. Anything I actually care about lives in the JSON, not in the wording.

The rules that matter:

• The orchestrator never writes code. It cannot edit, it cannot execute, and it cannot write anywhere on the system except its own scratch files. It gives instructions and nothing else.
• The orchestrator never writes documentation either. That’s Lambert’s job, and the separation is what keeps the record honest.
• Every coding step goes to the junior developer, no matter how the mission looks. There’s no such thing as a job small enough to skip the path.
• Every coding step is followed immediately by a review. Not at the end of the mission, immediately.
• Two junior rounds maximum. The junior fixes, the reviewer checks, twice. If the same critical issue is still there on the third pass, it escalates.
• The senior developer never appears in an initial plan. It arrives by escalation or it doesn’t arrive at all.

That last one exists because of what happened without it. Early on the orchestrator wanted to put the senior developer in the plan for everything, because on paper the stronger agent is obviously the better choice. It’s not, it’s just the more expensive one, and letting it into the plan quietly removes the entire review path. The two-round limit has a story behind it as well, involving a loop that ran overnight and a token budget I would rather have spent elsewhere. That one gets its own post.

The escalation numbers are the part I find most interesting. Across those 500-odd tasks, the senior developer has been called about thirty times. Everything else was resolved by a cheap agent doing the work and a strong agent telling it what was wrong with it.

Model Economics

When I first built this I had everything running on the strongest model available, on the reasoning that better is better. It got expensive very quickly, and it wasn’t noticeably better.

What it looks like now:

• Orchestrator: the strongest model, highest reasoning effort. Planning is the one place where paying more genuinely changes the outcome.
• Junior developer: Sonnet. It’s working inside a plan that has already been thought through, against conventions that are already written down.
• Code reviewer: strong model, mid-level effort.
• Senior developer: strong model, mid-level effort.
• Documentation writer: Haiku. It’s writing structured files to a known format.

The shape of it is that the cheap agent produces and the expensive agent judges. Judging is much cheaper than producing, because the reviewer is reading a diff rather than writing one, and a strong model reading a weaker model’s work catches most of what you were worried about.

The other thing worth saying is that the junior developer has got better over time, and not because the model changed. It got better because the harness kept learning what it tends to get wrong and writing that down where the agent would see it. Which brings me to the part I care about most.

The Task Log and the Improvement Loop

My biggest problem with agents was never capability. It was the black box.

I come from a background where if something goes wrong you go and look. You read the error log, you read the access log, you go back sixty days in the archive and you find the moment it changed. Apache doesn’t ask you to trust it. That was the thing I was missing, and early on I was also losing runs to models going up and down and connections dropping, which made it worse: something would fail halfway through and I had no way of knowing where halfway was.

So the state system came before anything clever.

Every mission gets an ID and its own folder, holding a context document and a progress document. The context is the context for that job. The progress document is the orchestrator’s plan, one line per step. As the work moves, the current step is written down: step three of ten, in progress. There’s a flight recorder alongside it, appended to as things happen, and an archive of finished missions.

The effect of that is I can kill the agent mid-task, come back in a completely new session tomorrow, and it reads the current mission file and picks up at step three. Continuity isn’t a matter of the agent remembering or of me being disciplined, it’s enforced by hooks that won’t let a step close without the state being written.

On top of the log sits the improvement loop, which I didn’t have at the start and would not now work without. In short: when an agent makes an assumption, misreads something or gets a fact wrong, that gets written to a findings file as a proposal. Patterns that show up across different codebases get written to a patterns file. I review those myself, daily, in small batches. The ones I approve go into a directives file that is loaded on every agent call, which means I can change how the agents behave without rewriting the agents. The ones I reject go nowhere.

The thing I like about it is that the rules come from what actually happened rather than from what I imagined might happen.

There’s a lot more to both of those than fits here. The state and log design is covered properly in I Didn’t Want a Smarter Agent, I Wanted a Visible One, and the move from markdown files to a SQL-backed memory service has its own post.

What’s Changed Since

It didn’t start as five agents. It started as two, an orchestrator and a junior developer, and then it grew.

Every time something didn’t work I added a piece. At its worst I had around ten agents, a pile of skills and about forty-five commands, and I had reached the point where I couldn’t tell you what would happen when I gave it a task. That’s the opposite of the thing I was trying to build.

So I stripped it back to the bone. Five agents, two commands, of which only one really matters, and the skills underneath them. What survived the cut was the mission log and the memory system, because that was the part carrying all the value.

There’s an honest gap between the public repo and what’s on my own machine, and it’s worth naming rather than hiding. The repo ships the crew with the character names and the mission-file architecture. My working setup uses plain role names, plus a few extra hands for dispatch, running tasks and research, and a slightly different set of state files. The repo is the harness with the personal furniture taken out, which is the version that’s useful to somebody else. Mine has six months of my own habits welded onto it.

The state layer is also mid-migration from markdown files to a small SQLite-backed service, which is a sibling project rather than part of the harness. Files were the right call to start with and stopped being the right call somewhere around search and history.

The Repo, and How to Take It Apart

Bishop is public and open source: github.com/djm56/bishop-harness. The memory service is at github.com/djm56/bishop-memory.

A small crew of AI agents with a chain of command, for Claude Code.

If you want a route in, read it in this order: the agent definitions to see who does what, the hard rules to see what they’re not allowed to do, and then the state files, which are where the actual design lives.

Please break it. Take out the escalation path and see what happens. Change the model on the reviewer and see whether you still trust the output. Throw away the character names. Take the mission log and leave the rest, that’s the part I’d keep if I could only keep one thing.

What I’d ask you to remember is that I built this for me. It fits the way I solve problems, the kind of work I do and the clients I do it for. I’m not putting it out as the right way to build a harness, and I don’t think there is a right way. If you get one idea out of it that changes how you work, that’s the whole point.

That’s Bishop. The rest of this series goes into each piece properly.

If you want help setting something like this up for your team, get in touch.