I Didn't Want a Smarter Agent, I Wanted a Visible One

I ended the last article by saying this one would be about memory.

It’s about the thing I built instead.

Because when I actually sat down with the problem, memory wasn’t what was bothering me. What was bothering me was that I couldn’t see anything.

The Black Box Problem

Think about how you normally work with an AI agent. You give it a task. Something happens. You get a result, or you don’t.

And that’s it. No history. No record of what it tried. No way to look back at last week and see what it kept getting wrong. No sense of what any of it cost you.

That’s a black box, and I’ve spent too many years in systems administration to be comfortable with a black box doing work on my behalf.

If a server misbehaves, I go and read the logs. I wanted the same thing here.

I didn’t want a smarter agent. I wanted a visible one.

So what I built isn’t really memory in the way people usually mean it. It’s closer to an event log and a task log. What was done, when, by which agent, what went wrong, and what I should do about it.

Every Task Gets an ID and a Folder

The first thing I did was stop letting tasks be vague.

Every task gets a proper ID, built from the date plus a number for that day. So a task ID tells me immediately when it happened and where it sat in that day’s work. Every task also gets its own folder.

Inside that folder, the task is broken into two documents:

Context — what this task is actually about, in the wider sense. The why, not the steps.
Progress — the breakdown of the work, ticked off as it goes.

The orchestrator builds the plan, and a plan is usually twelve or fourteen steps: junior dev, code reviewer, document writer, and so on. As the harness moves through it, the progress document gets updated. Not at the end. As it goes.

When the task finishes, it writes a done report. What it worked on, what issues it hit, where it got stuck, where it ended up in a loop.

Which means at any point I can open a folder and read exactly what happened.

The Active Task Pointer

There was one thing I thought about more than anything else in this part, and it’s the bit I’d recommend to anybody building something similar.

I wanted a task to be able to survive being interrupted.

Cancel an agent halfway through. Shut the machine down. Kill it because it’s clearly gone the wrong way. Whatever happens, when it comes back up it should be able to pick up where it left off instead of starting again from nothing.

So I added an active task: the ID of the task being worked on, and the step number inside it. That’s where the whole system actually started.

It sounds almost too simple to bother writing down. But combined with a progress document that’s kept current rather than written at the end, it’s the difference between an interruption costing you a few minutes and an interruption costing you the entire task.

Then It Turned Into an Improvement Loop

Here’s where it got more interesting than I expected.

Once you’ve got a proper record of every task, you’re sitting on something useful. You’re not just looking at what happened. You’re looking at what keeps happening.

So I built a task reviewer skill. It goes back through a completed task and pulls out two different kinds of thing.

Patterns

I work across some fairly extensive codebases. So if an agent is working in one place and notices a pattern that exists somewhere else in the codebase, that’s worth knowing.

It identifies the pattern and writes it into a patterns file for me to review later. Not to act on automatically. Just so it’s captured instead of noticed and forgotten.

Improvements

Then there’s an improvements file, and this is the one I’d argue is doing the real work.

Every time an agent got something wrong, assumed something it shouldn’t have, or did something that could have been done better, it gets recorded. Three parts, nothing more:

What the improvement is.
Where it happened.
What it recommends to fix it.

That’s it. No essay. An improvement that takes twenty minutes to read never gets read.

Nothing Gets In Without Me

This is the part I won’t compromise on.

Improvements don’t apply themselves. I go through them and I approve them. Once I’ve approved one, then it becomes part of the context the agents work with.

A self-improving system that improves itself without a human gate isn’t self-improving. It’s drifting, and you won’t notice until it’s already somewhere you didn’t want it to be.

Code Lessons and Methodology Lessons Are Different Things

The last piece is a conventions file, and it exists because I found I was mixing up two very different kinds of lesson.

Patterns and improvements are mostly about code. Conventions are about methodology.

So when I review the improvements, some of them clearly aren’t code problems at all. The fix isn’t a different function. The fix is a different instruction: something missing from how a skill or an agent was worded, a step it should have taken before it started, an assumption it shouldn’t have been allowed to make.

Those go into conventions, and the conventions file gets called in on every interaction.

Which means the agents get better at how they work, not just at what they know.

And Then the Conventions File Shrinks

Over time I take the conventions that have proven themselves and I word them back into the agents and skills directly.

So the conventions file gets smaller, and the agents get more specific.

I like that shape a lot. Conventions become a staging area rather than a growing pile: a lesson arrives, gets proven, and eventually stops being a note and becomes part of who that agent is.

Where It Started to Creak

I’ll be honest about the limits of this, because it worked well and it was easy to maintain, and both of those things were partly because I wasn’t running that many tasks.

I got to roughly a hundred tasks with OpenCode. About fifty with Claude.

And around there I started picking up real problems.

That’s the next article, and it’s the one where I finally do talk about memory properly, including why I ended up moving off markdown files and back to a SQL database. Which, after all these years, felt like a slightly funny place to arrive.

Both Versions Are in the Repo

There are two versions of this to look at.

The first version is the one described here, tightly integrated into the Bishop harness itself. The newer version is more structured and works a bit differently.

Go and look at both. Pull out the bits that fit how you work.

And if you take one thing from this article, take the boring one. Give your tasks IDs. Keep a progress document current. Write down what went wrong in three lines.

None of that is clever. It’s just the difference between an agent you’re trusting and an agent you can actually check.

Next: Bishop’s memory, properly this time. What broke once the task count climbed, why markdown files stopped being enough, and how the whole thing ended up in SQL.

I Didn’t Want a Smarter Agent, I Wanted a Visible One