The review you can't reproduce

A pull request lands, the tests pass, CI is green. Your reviewer reads it, sees nothing obviously wrong, and approves it in minutes.

Ask that reviewer two weeks later to change the same file and watch what happens. Sometimes it is fine. Sometimes there is a pause you have not seen before, and the honest version of it is: I approved that, but I could not have written it, and I am not sure I could tell you what it does when the input is empty.

That pause is the thing I want to talk about. Not because AI-assisted coding is bad, it is genuinely great, and I ship with it constantly. But because the pause is a management problem wearing an engineering costume, and most of the responses I see to it are aimed at the wrong layer of that problem.

The trade nobody made on purpose

Producing code is now near free while understanding code did not get cheaper at all. It still costs the same as it did in 2019 and earlier: attention, time, and that oh so specific discomfort of being stuck.

So the economics of review flipped. It used to be that reading a colleague’s diff was the cheapest way to understand a change, because writing it yourself was expensive. Now its more like writing is cheap and reading carefully is the expensive part. When the cheap thing and the responsible thing swap places, teams drift toward the cheap one naturally.

Nobody chooses to stop understanding the codebase. They choose, forty times a week, to approve something reasonable looking rather than reconstruct it.

— the actual failure mode

Forty small reasonable choices a week, across six engineers, for a year. That is the shape of it. There is no moment where a bad decision gets made, which is exactly why it does not show up in a retro or ever seem like a blaring issue in the beginning.

What it looks like before it costs you

The reason this is hard to manage is that every early symptom looks like good news.

Review latency drops. Throughput rises. The team’s velocity chart goes the direction you want. Meanwhile the signals that something is off are all soft, and they show up in places nobody instruments:

• Questions about a module route to one person, every time, and everyone has accepted that as the norm.
• Nobody volunteers to own the service that keeps paging. Not out of laziness. Out of an accurate read of their own understanding.
• Incident calls get longer at the diagnosis stage, even though the fix is fast once someone names the cause.
• New hires reach “shipping” quickly and “trusted with critical parts” never.
• In review, comments are about style and naming. They stopped being about the third case that the code does not handle.

That last one is the tell I would watch first. Style comments are what review produces when the reviewer has not built a model of the change. They are not lazy, they are what is left.

What does not work

Banning the tools. It costs you the velocity, which was real, and it does not survive contact with a deadline. What it actually produces is the same usage with less visibility, which is worse than either honest option.

Buying a course. Generic curriculum teaches someone else’s example project. The gap on your team is not “does not know Python.” It is “does not know why this repo has two HTTP clients and which one is deprecated.” No course covers your repository, because your repository is not in it.

Adding a review checklist. Checklists catch the things you already thought of. The whole problem is that the reviewer has not built the model that would let them notice the thing nobody thought of.

Writing more docs. Reasonable instinct, and worth doing for other reasons. But the documents describe the system as somebody understood it on the day they wrote it, and the repo moved this morning. Documentation is a lagging record of understanding, not a way to produce it.

What actually moves it

All of these are things a lead can start on Monday without buying anything.

Change what an approval means. Make it explicit, in writing, that approving a change asserts you could debug it at 2am. Not that you read it and nothing jumped out. That single redefinition does more than any process, because it converts a social act back into a technical claim. Expect review to get slower for a few weeks. That is the cost showing up on the correct line item.

Ask for the failure mode, not the summary. In review, the useful question is not “what does this do.” It is “what input breaks this.” A reviewer who has genuinely modelled the change can answer immediately. One who is pattern-matching cannot, and both of you learn that in about ten seconds without any awkwardness.

Rotate the second pair of hands. Whoever reviewed a change to an unfamiliar module gets the next change in that module. Not as punishment, as the cheapest way to convert a review into real knowledge. Two people who can confidently work in a file is a genuinely better operational position from only one, and it costs you an assignment decision.

Make “I could not rebuild that” a normal sentence. This is the one that requires you to go first. If a lead never says it, nobody below them will, and the information stops flowing upward entirely. I have found the most useful 1:1 question here is not “how are you doing,” it is: what did you ship this month that you could not rebuild from scratch? The answer is always interesting, and after doing it regularly, it stops being uncomfortable.

Treat incidents as a skills signal. The diagnosis phase of an outage is the most honest assessment your team will ever generate, and it is free. Not who caused it, but who could actually reason about it, and which parts of the system nobody could.

The uncomfortable part

None of this is a tooling decision, which is inconvenient for me, because I build a tool in this space. I left it out of the list above on purpose. The list works whether or not you ever hear from me again.

What I will say is that there is leverage in what you choose to measure. Completion percentages, courses finished, certificates, hours logged: none of those are real claims about capability. “This person changed the auth module unaided and here is what passed” is a claim about capability. If you are going to build one new signal this quarter, build that one, however that looks for you.

The teams that come out ahead in this era will not be the ones that used less AI or more AI. They will be the ones that stayed deliberate about which understanding to outsource, and noticing early when the answer to that starts drifting towards all of it.

Velocity you cannot explain is a loan, and the repayment schedule is set by your next outage.

— what the pause is telling you

If you lead a team and any of this sounded familiar, I would genuinely like to hear how it is showing up for you, especially if you have found something that works better than what I mention above. I am at lighthouse@inferhaven.com, and I answer everything.

— Ethan L.