AI code review is a strong pre-reviewer and a poor gatekeeper. On the bug classes where it's good — null paths, off-by-ones, leaked resources, missing await, inconsistent error handling — it's faster and more thorough than most humans on a Friday afternoon. On the classes that cause actual outages — cross-file invariants, concurrency, code that's internally consistent but does the wrong thing — it produces confident nonsense in exactly the same tone. Run it before human review, never instead of it, and manage it by one metric: the fraction of its comments that lead to a change.
The capability table, honestly
This is from about a year of running AI review on every PR across two codebases (a TypeScript web app and a Python data pipeline), plus what's consistent in published evaluations:
| Bug class | AI review performance |
|---|---|
| Null/undefined paths, unchecked returns | Reliably caught, often better than humans |
| Off-by-one, boundary conditions | Reliably caught in isolated functions |
Resource leaks, missing await, swallowed exceptions | Reliably caught |
| Injection-prone string building | Reliably caught |
| Cross-file invariant violations | Mostly missed — it reviews the diff, not the system |
| Race conditions, lock ordering | Missed or hallucinated in both directions |
| Wrong business logic that reads cleanly | Missed almost entirely |
| Performance regressions | Guessed at, rarely correct without profile data |
| Backwards compatibility, migration safety | Missed unless the contract is in the diff |
The pattern behind the table: AI review is excellent wherever the bug is visible inside the hunk plus a few hundred lines of context, and blind wherever correctness depends on knowledge that isn't in the diff — the other service, the database state, what the PM actually asked for. Larger context windows have moved this boundary less than you'd expect, because the reviewing model doesn't know which of your 400 files carries the invariant.
The false-positive budget decides everything
Review-fatigue economics are brutal and measurable. When more than roughly a third of AI comments are noise — style opinions dressed as bugs, "this could potentially be an issue" hedges, invented misuse of an API it half-remembers — engineers stop reading all of them within a few weeks, including the correct ones. At that point the tool has negative value: it adds scroll and subtracts trust.
So measure the action rate: comments that led to a code change divided by total comments. In my experience anything above ~50% keeps a team engaged; below ~30% the tool is on its way to being ignored. You move the number with configuration, not hope — suppress style and formatting entirely (linters own that), set the severity threshold high, and cap comments per PR at something like five. A reviewer that says one important thing gets read.
Make it prove the bug
The single best upgrade is a verification loop: findings must be demonstrated, not asserted. Concretely, pipe each candidate finding back through a step that writes a failing test or a reproducing input, runs it, and only surfaces the finding if the test is red. In my pipeline this cut surfaced findings by about half and pushed precision on the survivors above 80% — the discarded half was mostly hedges and hallucinated API misuse that couldn't survive contact with an interpreter. It costs extra tokens and ~1–3 minutes of CI time per PR, and it's worth both. The same evals-before-vibes discipline you'd apply to any LLM feature applies to the reviewer itself: keep a set of 50 known-bug diffs and 50 clean diffs from your own history, and score any prompt or model change against it before rolling it out.
Where it genuinely changes the process
Pre-commit self-review. The highest-value placement is before the PR exists — a local pass that catches the dumb stuff so human reviewers never see it. This is also where false positives are cheapest, since ignoring one costs a keystroke, not a thread. It slots naturally into an AI pair-programming workflow, where the same model that wrote the code should not be the only one reviewing it — use a different model or at least a fresh session for review.
Large mechanical refactors. A 3,000-line rename-and-move PR is where human attention fails and pattern-matching shines. AI review reliably spots the one call site where the migration was applied inconsistently.
Unfamiliar code. As orientation ("summarise what this PR changes and what could break"), it's consistently useful and can't really lie its way into damage, because you're using it to direct attention rather than to approve.
The 3am hotfix. A second set of eyes with no ego and no sleep requirement. Still not the approver — but the null check it catches at 3am is worth the subscription.
What it does to human reviewers
The quiet failure mode isn't the AI's comments; it's the rubber stamp. Once "the AI already reviewed it" becomes ambient, human review time drops — and it drops on exactly the design and intent questions the AI can't cover. Two counters work: make the AI pass a pre-review gate whose output the human reviewer can see is already handled (so their job is explicitly the remainder), and keep humans as the approver of record on every PR, no exceptions for green AI reviews — the standard set out in Google's code review guidelines still applies unchanged; the AI just moved which comments are left for humans to write. Tooling for dev-tools stacks makes it easy to wire the review bot as a required check; don't. Required checks should be deterministic.
What I'd do
Run AI review on every PR as a non-blocking commenter, style suppressed, five comments max, with a verification loop that discards findings it can't demonstrate. Track action rate weekly; if it falls under 30%, tighten the threshold rather than letting the team tune it out. Add a pre-commit local pass so humans never see the trivial findings. Keep merge authority with humans, and be explicit with the team about the table above — the tool is a tireless hunter of local bugs and a fluent liar about systemic ones, and a team that knows which is which gets value from both halves of that sentence.