AI Code Review

James Phoenix
James Phoenix

The bottleneck moved. Writing code is the cheap part now, and deciding whether to trust it is the job. Most of the advice I read about agentic code review treats that job as one problem with one answer: this is the reviewer tool you should buy, this is how many AI passes you should run, this is whether a human still needs to sign off. All of it falls apart the moment it touches a real team, because it skips the only variable that actually decides anything.

That variable is blast radius. What happens if this change is wrong, and how far does the damage spread? Risk is the core tenet of code review, and every other decision is downstream of it. How many reviewers, how adversarial they are, whether a human owns the merge: those are dials, and blast radius is the hand that sets them. Get the blast radius right and the rest of the review strategy falls out of it. Get it wrong and you will spend your most expensive attention in exactly the wrong place.

Everyone is reviewing a different thing

The reason “how should AI review code” has no single answer is that it is really four questions wearing a trench coat.

Marketing pages, copy, CMS content. Blast radius is near zero. A typo on a landing page is a five-minute fix that nobody remembers by lunch. I let the agent merge its own copy changes. Review here is theatre, and theatre is a tax on velocity with no payout.

Internal tooling and dashboards. Blast radius is a few annoyed colleagues and a Slack message. One automated reviewer, a passing CI build, ship it. The cost of being wrong is mild and the people affected can route around it.

Money paths. Payments, billing, auth, anything that moves a balance or grants access. Blast radius is real money and real trust, and both are hard to claw back once spent. This is where I crank everything to maximum. Multiple reviewers, a human owner on the gate, and no agent allowed to close the loop on itself.

Platform, infra, and shared libraries. Blast radius is fan-out. A bug here does not hit one screen, it hits every consumer of the library at once, and the failure shows up far from the change that caused it. The danger is diversity of failure modes, so the review has to be diverse too.

Same engineer, same agents, four completely different postures. The teams getting this right are not the ones with the fanciest reviewer in their CI/CD pipeline. They are the ones who matched the dial to the consequence.

A row of four semicircular gauges, one per domain. The needle sweeps further right and the fill grows as blast radius rises: Minimal for marketing and copy where the agent self-merges, Light for internal tooling with one AI reviewer and CI, Heavy for platform and infra with many heterogeneous reviewers, and Maximum for money paths with an adversarial stack and a human owner.
A row of four semicircular gauges, one per domain. The needle sweeps further right and the fill grows as blast radius rises: Minimal for marketing and copy where the agent self-merges, Light for internal tooling with one AI reviewer and CI, Heavy for platform and infra with many heterogeneous reviewers, and Maximum for money paths with an adversarial stack and a human owner.

Adversarial review is the lever that has moved the needle most

This is the change I would not give back. A single reviewer, human or AI, has one perspective and therefore one blind spot. The fix is adversarial review: multiple, differently-built critics whose job is to refute the change, not bless it.

On high-stakes code I run an actor-critic loop. One agent writes, another is told to assume the code is vulnerable and find every issue, and they loop until the critic cannot land a punch. On money paths I stack critics with different lenses: one hunting security holes, one checking the actual money math, one that just tries to reproduce a failure from the change. Each lens catches a class the others structurally miss.

The data is blunt about why this works. In Addy Osmani’s comparison of four AI reviewers across 617 distinct flagged locations, 93.4% were caught by exactly one of the four tools. Almost none by three. None at all by all four. That single statistic is the entire argument for adversarial review: the overlap between independent reviewers is tiny, so each additional differently-built critic is mostly net-new coverage, not redundancy. Run the same reviewer twice and you learn nothing. Run two different ones and the coverage compounds.

The cost is real. More tokens, more latency, more wall-clock before merge. Which is precisely why you spend it by blast radius. Three adversarial passes on the billing service is cheap insurance against a refund storm. Three adversarial passes on a blog post is pure waste. The lever only pays off when it is aimed at something that can actually hurt you.

Set the human dial by blast radius, not by guilt

As agents write more of the code, the instinct is that a human ought to read all of it. That instinct is wrong, and acting on it is how teams hurt themselves. Reading every diff does not scale, and worse, it spreads a fixed amount of human attention evenly across changes whose consequences are wildly unequal. You end up giving the dangerous PR the same tired eyes you just gave a dependency bump, and rubber-stamping both.

Leanpub Book

Read The Meta-Engineer

A practical book on building autonomous AI systems with Claude Code, context engineering, verification loops, and production harnesses.

Continuously updated
Claude Code + agentic systems
View Book

The move is to spend the scarce resource, human judgement, where blast radius is highest:

  • Auto-approve the low-blast-radius majority. A meaningful share of PRs, roughly a third in the experiments I have seen, are genuinely safe to merge on a risk classifier and a green build. Let them through.
  • Concentrate adversarial review and a named human owner on the load-bearing paths. Anything that moves money, grants access, or fans out across consumers gets a person on the gate who is accountable for being wrong.
  • Use conversational review for the large diffs that do matter, so the human reconstructs intent and impact instead of scrolling a wall of green and red and approving from fatigue.

How much human you keep is a dial, and you set it by what breaks if you are wrong, not by how guilty you feel about letting the machine merge.

The failure mode to design against

The thing that actually gets teams hurt is a confident “looks good” with nobody anywhere in the loop on a high-blast-radius change. The system’s certainty quietly becomes yours, and the truth is nobody understood the change at all. On a landing page that is completely fine. On a billing migration it is how you lose money and trust in a single deploy. Blast radius is the whole difference between those two outcomes, and it is the only thing that should decide how hard you review.

Writing got cheap. The cost of being wrong did not, and it was never uniform to begin with. Code review in the agentic era is not a quality ritual you apply evenly across every PR. It is risk management. Measure the blast radius first, then set the dial.

Related

Topics
Agent ReliabilityAi AgentsCode Review

Newsletter

Become a better AI engineer

Weekly deep dives on production AI systems, context engineering, and the patterns that compound. No fluff, no tutorials. Just what works.

Join 306K+ developers. No spam. Unsubscribe anytime.


More Insights

Cover Image for The Evolution of AI Coding up to 2026

The Evolution of AI Coding up to 2026

Software was always written in a loop: change something, run it, check the result, repeat. The last five years did not replace that loop. They kept widening the thing inside one turn of it, from a single line to a recurring workflow, and moved me one rung up the stack each time.

James Phoenix
James Phoenix
Cover Image for Loop Engineering

Loop Engineering

A loop is engineered, not launched. Every durable one has three parts: a trigger that wakes it, a bounded runner that does the work, and a gate that decides whether the work lands. The whole skill is matching each loop’s autonomy to whether you can write down what “done” means.

James Phoenix
James Phoenix