← All writing

Code is cheap. A codebase isn't.

On this page

“Just merge it, the AI will fix it.” It works. I’ve watched it work.

The ticket says: resending an invoice must not charge the card again. An agent’s fix:

async function sendInvoice(invoice: Invoice, skipCharge = false) {
  if (!skipCharge) {
    await chargeCard(invoice);
  }
  await emailInvoice(invoice);
}

sendInvoice(invoice);       // first send
sendInvoice(invoice, true); // resend

Bug fixed. Tests green. Ship it. It always works in the beginning.

I’ve been thinking about the purpose of PR reviews a lot lately, because several related trends landed on my timeline at once.

First, Jack Ellis, CTO of Fathom Analytics:

His advice, further down the thread: build first, test, rebuild, and engineer from there. I mostly agree with him. In practice, though, I see most engineers trying to refactor their way to the right solution instead of rebuilding.

Then Dylan Garcia, on what a review is even for now:

I’m all about two-way doors, so I relate. He does add a caveat: for migrations, DNS, or data leaving your control, slow down and do it properly.

And Uncle Bob, in a reply on whether you should read what your agents write:

His argument: with enough constraints (unit tests, Gherkin, mutation testing, coverage) it’s fine. Six weeks later he went further: he’s “starting to think that I’m over constraining them”.

The theme underneath all of them is how far we let agents go without a human in the loop. Each one drops a different human step.

I really like Addy Osmani’s framing in Software Factories, Light and Dark: a dark factory runs the agents with nobody reading the output. A lit factory is the same pipeline with “the lights left on where judgment lives”. His rule for choosing: you can only hand a loop as much autonomy as you can cheaply and reliably verify. This post is me working out whether PRs are one of the places the lights stay on. For now, at least.

Where I agree with all of them: it works.

See, it always worked in the beginning. Every startup I’ve watched shipped fast at first on a codebase nobody defended. The ones who had somebody around who’d seen it before found a way through. The rest sank into the same tar pit, somewhere between month six and month eighteen. I’ve spent over half my career in startups, the rest in government and consulting. I tell technical people to move faster at least as often as I tell them to slow down, because speed to market beats almost everything. I also know what the bill looks like later. I think you can have both, as long as someone on the team can smell the tar before they’re in it.

Uncle Bob charted the tar pit in the first chapter of Clean Architecture: productivity per engineer falling release after release, heading for zero, while the cost of every line climbs. In a lot of ways that book is what the job of a software developer was to me: standing up for standards and architecture, and weighing short-term gain against long-term velocity on every change.

So here’s what keeps me up at night, as the person running engineering teams and responsible for where the product is in a year. I’m enjoying the velocity gains as much as anyone. I’m also the one accountable when developers quit, when velocity drops, and when we no longer have four nines of uptime. And when a feature is late because it lives in a corner of the platform that a handful of customers use, an agent last touched a year ago, and nobody on the team wants to open. Sadly the conversation online only seems to be about the first six months.

To figure out where I land, I listed the reasons we use(d) PR reviews in the first place. When I did, the reasons sorted roughly into two buckets: what you can verify today, and what you pay for in month six.

Bucket one: what you can verify today

Catching bugs. For correctness, meaning the manual QA, I’ve always expected full ownership from the developer opening the PR. Test it by hand, or write the tests that prove it. If developers own correctness and the infrastructure is built for recovery (feature flags, instant rollbacks), this is a good candidate to hand to AI. My own feeling is that AI matches human review here, and sometimes beats it, as long as you use frontier models, add an adversarial verification pass, and give the agent a codebase it can run and check on its own: compare screenshots, read logs, and so on.

Is the direction right. A comment nobody wants on their PR. You spent two days on it and you feel great. Then someone drops in with “why is this needed?” or “why didn’t you use service X?” Knowledge is spread across a company, and sometimes someone knows a better way. That used to be expensive: follow-up tickets, ship, iterate, or two days in the bin. Now that code is cheap, it should mean closing the PR and starting over, the rebuild from Jack’s thread. And a lot of it is catchable before the PR exists. An agent with the whole repo loaded can spot “service X already does this”. “Why is this needed” still takes a human, so here I go a little against Jack: some planning up front, pair prompting, a quick “what are you about to build?” before the first line. That won’t catch all of it. It catches enough that the review isn’t the place for it anymore.

SOC 2. Honorable mention. Most SOC 2 compliant companies probably have a control that says all merged code is reviewed and approved by another developer. Strictly, SOC 2 doesn’t require it. Your controls do, because you wrote them that way. I do believe a model review will be enough eventually.

That’s bucket one. Everything in it you can check before the merge. If that were the only purpose of PRs, I’d agree. Two-way doors. Ship it.

Bucket two: what you pay for in month six

Nothing here fails a test today. All of it shows up later, in the code or in the people who have to work in it.

Standards. In theory solved. Linters and formatters handle the mechanical ones. Write the rule once in a skill or an AGENTS.md, and the models are great at copying whatever pattern already exists in the repo. In practice it still comes up a lot. Part of it is mechanical: models are non-deterministic, they don’t follow your rules 100% of the time, and context bloats. The bigger part is that the standards that matter most are the hardest to put into words. I might be biased here. I’ve always thought rules that make you think beat hard rules. Rules are for the guidance of wise men and the obedience of fools, as the saying goes. Most codebases I’ve enjoyed working in ran on the first half of that sentence. More on what that means for the rule file below.

Honesty, or accountability. I love code. I care about it more than I should at times. But I slip too. A Product Owner or an Account Manager asks if you can get this out quickly. The only thing between you and being celebrated for saving the day is a unit test. “I’ll add the test later.” Narrator: later never came. Once code is merged, new features beat tech debt every time. A PR is a place where you get called out.

Mentoring and growth. PRs do double duty here. On one hand they’re a tool for me to scale myself. I can squeeze a review in between meetings and give a developer real feedback through comments and alternative approaches. On the other, they’re one of many data points I use to check on growth. A new hire might get thirty comments on their first PR, a month later five, and so on. It’s just a number, but it helps paint a picture.

A conversation with a new hire a while ago stuck with me. Using AI, they were able to contribute from day one to a codebase in a language they had little experience with. Three months in, they told me their confidence in their knowledge of the codebase hadn’t grown much since. That’s worrying. I’ll go out on a limb: no developer has ever felt they drastically improved because a review bot left a comment on their PR, even a correct one. And I won’t count bot findings as a metric of anyone’s growth.

Nobody understands the code anymore

Knowledge sharing, or what’s left of it. The classic argument: at least one person besides the author knows this code exists, how it works, what it touches. Bus factor two instead of one. I always had my doubts about how much knowledge a review really transfers, but I accepted it and never pushed on it. Now I don’t think it can be real anymore, because I’m not sure the developer who opened the PR understands it either.

Addy calls it comprehension debt: the gap between how much code exists and how much of it any human still understands. I’m not sure how to track it. The code is so clearly not written for the next human. Auto-generated PR descriptions that explain the what and the how and never the why. Disjointed stacks of fifteen PRs. The code isn’t wrong, necessarily. Just alien. Every abstraction one notch too general, nothing simple.

And I don’t think developers are good at iterating it back down. To simplify code you have to understand it, and nothing gets you there like having written it. I’ve caught myself spending longer shrinking a stack back down than writing the feature would have taken. How bad this gets depends on the size of the change. A five-line bug fix is fine. A user story is some work. A new subsystem is where I lose days. None of that is new with AI. What changed is people reach for the subsystem-sized change more often now, because it’s easy and code is cheap.

Underneath the stack problem is a bigger one. To reason about a system at all, to say whether a change to the data model is right, I have to truly understand the code. And writing it is what gave me that. Reading the diff doesn’t get me there. Reviewing doesn’t. Writing the spec doesn’t, and grilling myself on the plan before a line exists doesn’t either, however well done. Yes, I generate diagrams and interactive HTML pages for myself, and of course they help. But quiz me on the feature two weeks later and you’ll see how rotten my brain has gotten. Compared to this, TikTok is a workout for the mind.

What AI writes, AI can’t fix

Three examples, all simplified, all the same shape. The outcome is right, exactly what the “only the outcome matters” crowd asks for. The tests are green. And the agent that wrote it won’t fix it on its own, because from where it stands nothing is broken. It found the change that satisfies the request. It never stepped back to ask whether the request exposed a problem with how we modelled the thing. I haven’t noticed a difference between Opus 4.5 and Fable here.

The first one you’ve already seen: the invoice flag from the top of this post. Bug fixed, test passes, outcome correct. And the function now has two modes and a flag to pick between them. Keep extending it this way, and by month six it has four flags and nobody knows which combinations are valid. The better solution:

async function issueInvoice(invoice: Invoice) {
  await chargeCard(invoice);
  await emailInvoice(invoice);
}

async function resendInvoice(invoice: Invoice) {
  await emailInvoice(invoice);
}

Same outcome. No flag. The next person who needs “email without charging” calls the function instead of adding a fifth boolean.

Second pattern: the “load-bearing” comment.

await updateUser(id, patch);

// NOTE: this delay is load-bearing. Without it the next read returns
// the old user. Do not remove.
await new Promise((resolve) => setTimeout(resolve, 200));

return getUser(id);

That’s a stale read with a note attached. The comment even sounds responsible. It’s the agent documenting the bug instead of fixing it, because the outcome was met: “the updated user comes back”. The fix is somewhere else: find out why the read is stale, then return the updated row from updateUser or invalidate whatever is caching it. The agent never went looking for where the staleness came from, because the test was already green.

Hot tip: don’t tell the AI to never leave comments. A comment that explains a workaround is a flag on where the code could have been better, and the AI won’t get there without you.

Third: the rule that got copy-pasted. Support staff can now manage users, so isAdmin || isSupport appears at six call sites. Nobody introduces canManageUsers. Every call site is correct. The rule lives in six places, and the seventh will get it wrong.

The admission that goes with this: I’ve taught workshops on it. I’ve taught best practices and put systems in place on teams of every skill level, background, and mentality. And I never managed to systemize how I think about architecture to a level I’m truly happy with. I can write the rules down, and I did, every time. And every time the rules covered about half of it. The other half was engaging with the specific problem in front of you and leaving room for best judgement. The wise-men half.

So when someone tells me to just put it in the AGENTS.md, I believe them. I’ve also been trying to put it in the AGENTS.md for humans for as long as I can remember. The half that wouldn’t go in is the half in the three examples above. And it’s not for lack of material. Every one of the three has a rule with a name. The boolean flag is the flag argument Uncle Bob calls “a truly terrible practice” in Clean Code, chapter three. The load-bearing comment is chapter four: don’t comment bad code, rewrite it. The copy-pasted permission is chapter seventeen, duplication. The models have read these books, or everything ever written about them. Feed them to the agent and the three examples above are still what comes out, because the books were never the rules either. They were the judgement, explained.

And past the principles is the data model, which the books cover least and matters most. DDD has been around for twenty years, and spec-driven design is the same idea with a new name. AI hasn’t solved the hard part of either, which is deciding what the domain actually is. A whole other post.

None of these fail a test. All of them show up in the next ticket, and the one after that. This is exactly the second chapter of Clean Architecture, the section literally called “Fight for the Architecture”. The business will always pick the urgent feature over the important structure, so it falls on developers to defend the architecture. If architecture loses every argument, the system gets slower to change until it can’t change at all. “Velocity isn’t a problem anymore” is a claim about this week’s velocity. Architecture is about month six.

And the thing that took me longest to see: code quality used to be enforced by self-interest. The person adding the boolean flag was going to be the person maintaining it. The agent never has to live in the codebase it leaves behind.

What I’m trying currently

Not a framework. A list of things I’m putting more focus on, all aimed at bucket two. Bucket one I’ve handed off. These sit on top of a setup that’s already pretty AI native: review skills, agents on ticket triage, a self-improvement loop.

Units, not features. Features run away from you. One giant grill-me session and then “build the feature” tends to produce the fifteen-PR stack because it was asked for a feature. I’m going back to something closer to how we worked early on with Cursor and Windsurf in the editor: one piece at a time. Build a unit, verify the interface, build the next. Slower per feature. Perhaps faster overall. Especially after a prototyping session or two first, so the units are the right ones. That’s Jack’s “build first, observe, engineer from there”, and I think he’s right. The engineering step is the one I don’t want to skip.

The trick is finding the threshold. Below it, small and low blast radius, the agent one-shots the change and I don’t look twice. Above it, we hand-craft the shape and let the agent fill it in. I don’t want to over-correct here. I know I’m the bottleneck. I’m hoping to find the balance.

Zero AI-generated PR text. Write it yourself: what problem it solves, why this approach, and how you checked it. If you can’t, you don’t understand it yet, and finding that out is the point. It’ll also be shorter. As you may have noticed from this 3,000-word essay, that’s something I desperately need to work on. Pascal, in a letter from 1656: “The present letter is a very long one, simply because I had no leisure to make it shorter.” My clankers don’t get leisure, so it’s on me. Then hand that description to the AI reviewer along with the diff. Now it can tell you when the code doesn’t do what you said it does.

Read for patterns like these three. The bot reads for bugs. I read for a flag that should have been two functions, a comment that’s documenting a bug, a rule copied to six places. It’s a shorter review than the one I used to do, and a harder one, because the diff is fine. The problem is what the diff should have been.

For the people half of bucket two, I don’t have a practice yet, beyond keeping human comments in the review. That one stays open.

Where I land, for now

Bucket one: lights off.

Bucket two: lights on, but I’ve started dimming them. At least until I’m confident it’s feasible long term.

But how would I even know? We started tracking early on: output, velocity, bugs, and a regular dev check-in. Compared to the days of no AI or autocomplete-style AI, velocity is up about 4x across our productivity metrics. Fewer outages. Long-standing issues closed, a lot of tech debt addressed, and bugs under our zero-bug policy get fixed on average in less than eight hours instead of seven days. Business KPIs match the trend. Very outcome-focused again, I know, and I still don’t think it captures the essence. Can you really quantify quality? Maybe it doesn’t matter. What those numbers do buy me is the right to be cutting edge instead of bleeding edge. At least while we’re still figuring out the right path forward.

But hey… maybe none of this matters. Maybe I’m focused on the wrong thing. Maybe the right move is to let go completely and let the tooling take me. Some days I think that. I know the reply, too: if code is cheap, the month-six cleanup is cheap as well. Except I’m not so sure it is. The fundamentals didn’t stop mattering, they moved to month six, and nobody is writing about month six. But I’m the one accountable for that feature, and that uptime, and I’m not at that point yet. So I’d ask one thing. Am I willing to bet the company on it? Are you? Because if it doesn’t work out, the person who told you coding was solved won’t be there to help you unfuck your codebase.