AI Code Review Is Failing and What To Do About It.

AI code review is failing for a structural reason rather than a tooling one. The pull request presents changes as an alphabetized file list, a format that works at ten lines and collapses at five thousand, and five thousand lines is now about an hour of agent output. Across nearly 28 million workflows, CircleCI recorded feature-branch throughput up 15% year over year against main-branch throughput down 7%.

Key takeaways

• The pull request was built to hold a stranger's contribution at arm's length. Commercial teams adopted it anyway and hung their whole process off it.
• Main-branch success sits at 70.8%, the lowest in five years. The median team burns 3.9 workflow runs per merge; the top twenty organizations burn 1.3.
• Validation has to move twice. Into the agent's inner loop, and up into the spec, where the misalignment failures get decided.
• Code that compiles and does what it was told can still be wrong about the system it landed in. Review never reaches that.

The pull request was built to keep strangers out

Rob Zuber has been CTO of CircleCI for twelve years, a good seat for the entire lifespan of the modern pull request. Halfway into recording Stacked Sessions, he made the argument that stuck with me.

"The PR was a great way to keep people from contributing code to your codebase. I don't trust you because I've never met you. Please fill out this form. And I will decide later whether I'm going to bother reading it, let alone read any of your code."

Against a stranger on the internet, that distrust was appropriate. Commercial teams picked up the artifact anyway, hung protected branches and named approval lists off it, and tolerated the overhead for two decades because, as Rob puts it, "most of those other things that we use, let's use the PR as an example, came into play based on an assumption that writing software was really expensive." That assumption stopped holding about eighteen months ago. Generating a change is now cheap enough that the gates cost more than the mistakes they catch.

What 28 million workflows say about AI code review volume

CircleCI's 2026 State of Software Delivery report covers nearly 28 million CI workflows across 22,000 organizations. Average throughput grew 59% year over year while median main-branch throughput fell 7%.

"We're seeing a large uptick in branch builds, meaning people are trying to get stuff to build, but not really an uptick in main. So they're not releasing; they're just building more, which is bad."

Main-branch success sits at 70.8%, the lowest in five years against a healthy threshold of 90%, so nearly 3 in 10 merge attempts fail. The top 5% of teams nearly doubled daily workflow runs while the bottom quartile showed no measurable increase.

The Q2 2026 Pulse report adds a metric worth stealing. Merge Efficiency Ratio counts the runs burned per change landed on main: 3.9 for the median team, 1.3 for the top twenty organizations, and the gap is rework priced in build minutes and tokens.

What should replace AI code review

Human review caught some things, inconsistently, and we oversold it to ourselves.

"By the time I get to the part that is interesting, I'm already trying to figure out what I'm going to make for dinner. How many reasons do you need to understand that we were not good and consistent about evaluating the quality of software by reading it after the fact? But you know what's really good at doing large scale data analysis is machines."

Metrics vendors flagged PR review time as the top bottleneck well before AI touched the pipeline. Volume did not create this problem. It removed our ability to ignore it. The artifact gets stranger still once you notice that both the writer and the reader are increasingly agents, talking to each other through CLIs we built around a web form.

Rob is not arguing for less validation. He is arguing that we aimed it at the wrong target.

"Don't say this 5,000-line change is too risky, Rob; go review it. Say these four lines are risky for this reason. Rob, go read these four lines. Everything else is clear to go."

Machines perform large-scale analysis and escalate only what exceeds a risk threshold. Hold onto that shape, because it comes back.

Validation has to move left twice

"If your linter is failing in a CI build, I feel like you've made a mistake."

Rob's team wired CI-grade checks into the agent loop and found their local tooling was configured differently from CI, so work passed locally and failed in the build. The fix was a CI-like environment inside the inner loop, with a hard gate on the agent's definition of done. CircleCI models a 50-developer team at roughly $900,000 in annual delivery cost and puts $700,000 of it in reach.

That first shift handles what a machine can check against a known standard. The second is the one no validation in the loop reaches, and Thoughtworks named it precisely. AI-generated code is often syntactically correct but contextually misaligned, failing against shared state and legacy API contracts.

That failure mode deserves a careful read. The code compiles, passes tests, and does exactly what it was told. It is wrong about the system it landed in, or about what the customer needed, and both were decided before the agent started. A meaningful share of that 3 in 10 failure rate is code which faithfully implemented an underspecified request, and half of product and engineering teams already report ticket quality causing drag.

Where the second shift lands

Allstacks Product Studio is a workspace for product and engineering to define specs grounded in the codebase, the customer, and the delivery history. Adversarial AI reviewers stress-test the spec before engineering touches it, covering security, QA strategy, architecture, and feasibility. Each returns findings attached to specific parts of the spec with the reasoning shown, which is Rob's four-risky-lines pattern applied one layer up. The reviewers have brownfield awareness because the failures Thoughtworks describes occur in existing software.

CircleCI puts $700,000 a year on moving validation into the inner loop. Specs that eliminate one rework cycle per sprint return the platform investment in under a quarter. Both numbers describe work that was done and then repeated, and DORA's ROI data consistently shows that returns come from practices rather than purchases.

How to fix your code review process this quarter

Pull your branch-to-main ratio. CircleCI's median is 3.9 runs per merge and the best teams run 1.3, so you have a benchmark. If branch volume climbed and merges did not, you bought throughput you are not shipping.

Move one check from CI into the agent loop this sprint, starting with linting and formatting, since a linter failure in CI is pure waste and the fix is configuration rather than architecture.

Then sort your failures. Some were mechanically wrong, and autonomous validation fixes those. The rest were built faithfully from a wrong request, which is a spec problem invisible to code review by definition.

Rob's closing position is a method, not a verdict. "Back up, go up a level, and ask why we were doing these things? And with a new set of tools and a new set of constraints, what would be the best way to do these things? We'll get way better outcomes than 'how can I optimize PR review.'" Run that on your review process before you buy another tool for it.

Listen to the full episode of Stacked Sessions with Rob Zuber. Allstacks reads CircleCI pipeline data through our CircleCI integration.

Jeff Keyes is Field CTO & Product Leader at Allstacks and hosts Stacked Sessions.

Allstacks Product Studio is what product and engineering use to shape the inputs grounded in reality (ideas, specs) so that outputs (AI code) come out with fewer issues. Sign up for Free!