How to Measure Code Review Quality (5 Metrics That Are Not Vanity)

Direct answer

Most teams measure review throughput and call it quality. Five metrics survive contact with Goodhart’s law: review latency at p50 and p90, scrutiny density (inline review threads per 100 changed lines), defect escape rate, rework rate, and review participation spread. Our 2026 baselines: open to first review is p50 8.7 hours and p90 212 hours, and the median pull request spends 52.7% of its open-to-merge life waiting for the first human. Every metric here turns destructive the moment it ranks individuals.

How do teams measure code review quality?

Direct answer: Badly, and mostly by accident. The common dashboard shows pull requests merged, comments posted and time to merge, all of which measure throughput. Quality requires measuring whether review changed the outcome: defect escape rate, rework rate and scrutiny density. Use five metrics and never rank individuals on any of them. Measured 2026-10-18.

Before the metrics, the baselines, because our own data is the thing nobody else has.

Dataset 1: PR size and scrutiny. 2,896 human-authored merged pull requests across 10 major public repositories, 104 bot authors removed, created 17 April to 17 October 2026, harvested from the GitHub GraphQL API on 2026-10-18.

Pull request size	Inline review threads per 100 changed lines	Mean threads per PR	Share with zero inline comments
Under 50 lines	1.55	0.28	High
Over 1,000 lines	0.085	3.15	52%

That is an 18-fold drop in scrutiny density across the size range. The honest counterpoint, which we published alongside it: absolute comment counts rise with size (Spearman rho +0.355), so large pull requests are not ignored. They are reviewed far less per line. Full method, per-repo spread and limits in how big should a pull request be.

Dataset 2: review latency. 2,189 human-authored merged pull requests across 12 repositories, bot authors and bot reviewers removed, self-reviews excluded, same harvest date.

Metric	p50	p75	p90	p95	p99
Open to first review (hours)	8.7	66	212	482	~1,300
First review to merge (hours)	6.7	-	-	-	-
First approval to merge (hours)	1.1	-	-	-	-

The median pull request spends 52.7% of its open-to-merge life waiting for the first human to look at it. Also: 59.2% of merged pull requests had exactly one approver, 14.5% had two, mean 1.08. There is a 9x spread in median time-to-first-review between sub-50-line and 1,000-plus-line pull requests. Full method and disclosed limits in pull request review time.

Disclosed limits on both datasets, because they bound every conclusion below: only merged pull requests are in frame, which biases latency downward by excluding abandoned work. Kubernetes and Rust are Prow and bors artifacts and were excluded from the approver analysis. These are public repositories with volunteer reviewers, so the latency tail is worse than a funded team’s would be.

Here is the metric set, with the failure mode stated for each, because a metric without a stated failure mode is a trap.

Metric	What it measures	How to compute	Our 2026 reference point	Failure mode when gamed
Review latency percentiles	How long work waits, not how long review takes	Hours from PR open to first human review comment. Report p50 and p90	p50 8.7h, p90 212h	Reviewers skim to clear the queue. Pair with scrutiny density
Scrutiny density	Attention per unit of change	Distinct inline review threads / (additions + deletions) x 100	1.55 under 50 lines, 0.085 over 1,000	Nitpicking inflates it cheaply. Pair with rework rate
Defect escape rate	Whether review caught what mattered	Production defects traceable to a reviewed change / total reviewed changes, per period	Not measurable from public data	Teams stop filing bugs. Measure from incidents, not tickets
Rework rate	How much review changed the code	Share of merged PRs with at least one post-review commit that is not a rebase or a lint fix	Not measurable from public data	Authors squash before merge to hide it. Measure pre-squash
Review participation spread	Whether knowledge is distributed	Gini coefficient or top-reviewer share of review threads per repository per quarter	59.2% of PRs had exactly one approver	Round-robin assignment produces fake spread. Pair with latency

Throughput is not quality, and the confusion is expensive

Direct answer: Throughput metrics (PRs merged, comments posted, time to merge) measure how fast the pipe moves. Quality metrics (defect escape rate, rework rate, scrutiny density) measure whether review changed the outcome. A team can double throughput and halve quality simultaneously, and most review dashboards cannot tell the difference. Verified 2026-10-18.

The structural reason the two get conflated is that throughput is cheap to instrument. Every forge API hands you merge counts and comment counts for free. Nothing hands you “would this bug have shipped without review”, which is the only question that defines quality.

The clearest demonstration is in our own numbers. Large pull requests merge with fewer comments per line and fewer approvers, and they merge fast once someone starts: first-approval-to-merge is 1.1 hours at p50. By any throughput measure, a team shipping 1,500-line pull requests with one approver and two comments looks excellent. By any quality measure it is a team with no review at all.

So the rule: never report a throughput metric without the quality counter-metric next to it. Latency pairs with scrutiny density. Merge volume pairs with rework rate. Comment count pairs with nothing, which is why it should not be on the dashboard. More on the behavioural side in pull request review best practices.

Metric 1: review latency percentiles, and why p90 beats the median

Direct answer: Measure hours from pull request open to the first human review comment, and report p50 alongside p90. In our 2026 data p50 is 8.7 hours and p90 is 212 hours, so a healthy-looking median conceals one pull request in ten waiting over a week. If you publish one number, publish p90. Measured 2026-10-18.

The median describes the pull requests nobody worries about. The tail is where the damage is:

• Context decay. The author has moved to other work. Returning to a pull request after a week costs a re-read of your own diff.
• Rebase compounding. A pull request open for 200 hours on an active repository accumulates conflicts that are themselves a defect source.
• Reviewer substitution. The person who should have reviewed it is no longer available, so someone with less context does.

What to instrument:

1. Open to first human review comment. Not first bot comment. If you have an AI reviewer, you must exclude it or the metric becomes a measurement of your bot’s uptime.
2. First review to merge, separately. Ours is 6.7 hours at p50, against 8.7 hours of waiting beforehand. The review itself is not the bottleneck.
3. Open to merge, as the envelope. Then compute waiting share: ours is 52.7% of total life spent waiting for the first human.

The leverage is routing, not effort. A team that tells reviewers to “be faster” is optimising the 6.7 hours. A team that fixes CODEOWNERS, caps per-reviewer queue depth and alerts on unassigned pull requests is optimising the 8.7. LinearB and Graphite both surface per-reviewer queue depth and time-to-first-review, which is the view you need before buying anything. Tactics in reduce code review time and how to make code reviews faster.

One warning on targets: do not import ours. Public repositories have volunteer reviewers and that inflates the tail relative to a funded team. Use our numbers as a reference point and set your own service level from your own distribution.

Metric 2: scrutiny density, which we are effectively introducing

Direct answer: Scrutiny density is distinct inline review threads divided by changed lines, times 100. It measures attention per unit of change, which no standard review metric does. Our 2026 baseline falls from 1.55 on pull requests under 50 lines to 0.085 on pull requests over 1,000. Treat it as a first reference point, not an industry standard. Measured 2026-10-18.

We have not found this metric defined and baselined anywhere else, so here is the precise definition rather than a vague one.

Definition. For a pull request P:

scrutiny_density(P) = (distinct inline review threads on P / (additions + deletions)) * 100

Computation rules, all of which change the number materially:

1. Count distinct threads, not individual comments. A thread with eight replies is one unit of attention, not eight. Counting replies rewards argument.
2. Count inline threads only. Top-level pull request comments (“LGTM”, “can you rebase”) are not scrutiny of code.
3. Exclude bot authors. On a repository with an AI reviewer, bot threads will dominate and the metric stops measuring human attention. If you want a machine-scrutiny number, compute it separately.
4. Use additions plus deletions, not net change. A 500-line deletion is 500 lines a human should have looked at.
5. Exclude generated and vendored paths from the denominator. In our sample of 80 merged pull requests over 1,000 changed lines, 42.1% of all changed lines sat in generated, vendored or lockfile paths. Leaving them in the denominator makes any team shipping lockfile updates look negligent.
6. Aggregate by size band, never as a single repository average. The metric varies 18-fold across the size range, so a single average is dominated by your size mix rather than by your review behaviour.

Failure modes, stated honestly:

• Nits inflate it for free. A reviewer who posts twelve import-ordering comments scores higher than one who posts one architectural objection. This is the dominant risk and the reason it must be paired with rework rate. See what is a nitpick in PR reviews.
• It penalises good small pull requests. A clean 20-line change that genuinely needs no comment scores zero. Density is a distribution metric, not a per-PR judgement.
• It is confounded by author seniority. A senior engineer’s diffs legitimately attract fewer comments.
• It says nothing about whether the comments were right. CRJudgeBench (arXiv 2609.37216) exists precisely because plausible-but-invalid review comments are a real category.
• Our baseline is from public repositories. Open-source review norms are not corporate review norms.

Used correctly, it answers a question nothing else does: is the attention your team spends proportional to the change it is spending it on? In our data, the answer across the industry is emphatically no.

Metric 3: defect escape rate

Direct answer: Count production defects traceable to a change that went through review, divided by total reviewed changes, per period. It is the only metric that directly measures whether review worked. It is also lagging, noisy at small volumes, and corruptible the moment anyone is judged on it. Verified 2026-10-18.

Instrumentation that works:

• Source from incidents and production error tracking, not from the bug tracker. Bug trackers measure reporting behaviour. If defect escape rate becomes a target, the first casualty is ticket creation.
• Trace via the deploy, not via the author. Link the incident to the release, the release to the merge commits, the merge commits to the reviewed pull requests. Author attribution is both inaccurate and politically toxic.
• Window it at a quarter minimum. Monthly numbers on a normal team are statistical noise.
• Segment by severity. A quarter with three cosmetic escapes and a quarter with one data-loss escape are not the same quarter.

The measurement problem nobody solves: you cannot observe the counterfactual. Defect escape rate tells you what review missed. It never tells you what review caught, because caught defects do not exist as production events. This is why review ROI arguments are so weak, and why the published figures are so unreliable.

On ROI specifically: do not run a single loaded-cost figure. BLS data from March 2026 puts private-industry total compensation at $46.60 per hour; third-party models put senior US engineers at $130 to $170 per hour loaded. Run your model at $50, $75, $100 and $150 and let the reader pick. Cost modelling for the tooling side is in what does AI code review cost.

Metric 4: rework rate

Direct answer: The share of merged pull requests with at least one substantive post-review commit, excluding rebases, merge commits and lint-only fixes. It is the closest available proxy for “review changed the code”. Measure it pre-squash or you will measure nothing. Verified 2026-10-18.

Rework rate is the counter-metric that makes scrutiny density safe. High density with low rework means your reviewers are generating comments that do not change code: nits, questions, and approval theatre. High density with high rework means review is doing work.

Classification rules, which are where this metric lives or dies:

Counts as rework: a code change in response to a review thread; a test added after a reviewer asked; a design change after an architectural objection.

Does not count: rebases and merge commits; formatter and linter fixes; typo fixes in comments or documentation; changes the author made for their own reasons while waiting.

The squash problem. Teams that squash-merge destroy the evidence. Capture commit history from the pull request’s commit list via the API before the merge, or compute rework at review-thread level instead: the share of inline threads that were followed by a push touching the same file. That variant is noisier but survives squashing.

Target range. We do not have a public baseline for this, and we are not going to invent one. What we can say is directional: a repository where fewer than 10 percent of merged pull requests show substantive rework is running review as a formality, and a repository above roughly 60 percent probably has a problem upstream of review, in design or in pull request size rather than in the review step itself.

Metric 5: review participation spread

Direct answer: Measure the share of review threads authored by the top reviewer per repository per quarter, or compute a Gini coefficient across reviewers. Concentration is a bus-factor risk and a latency cause. In our 2026 data, 59.2% of merged pull requests had exactly one approver and the mean was 1.08. Measured 2026-10-18.

The single-approver finding is the one to sit with. Nearly six in ten merged pull requests in major public repositories were approved by exactly one person. That is the real review topology, not the two-reviewer policy most organisations write down.

It matters for three reasons:

1. Bus factor. If one person authors 60 percent of review threads in a repository, that repository has one reviewer and a succession plan problem.
2. Latency. Concentration causes the p90 tail. A single reviewer on holiday is a 200-hour wait.
3. Separation of duties. This is where the AI question gets sharp. GitHub Copilot code review can submit an approving review that satisfies a repository’s required-approval rule the same way a teammate’s approval would (GitHub documentation, verified 2026-10-18). It is off by default and configurable at enterprise, organisation and repository level. A machine that can satisfy a required approval weakens a single-approval rule rather than strengthening it. GitHub’s required_approving_review_count range is 0 to 6; GitLab’s maximum is 100. By contrast, Anthropic’s Code Review never approves or blocks, which is the safer default for a team relying on approval counts as a control. More on reviewer counts in how many reviewers should a pull request have.

Measure it per repository, per quarter, on trend. Never per person.

The three anti-patterns

Direct answer: Comment count as a proxy for rigour, approval speed as a proxy for efficiency, and any per-developer leaderboard. All three are actively destructive rather than merely useless, because each one has a cheap gaming strategy that makes review worse while the number improves. Verified 2026-10-18.

Anti-pattern	Why it looks reasonable	What it actually produces
Comment count as rigour	More feedback looks like more care	Nitpicking, because nits are cheap to author and count identically to substantive findings. Once an AI reviewer is posting, the metric measures nothing at all
Approval speed as efficiency	Fast approvals look like a healthy pipeline	Rubber-stamping. Our data already shows first-approval-to-merge at 1.1 hours at p50, so there is almost no slack left to optimise here, and squeezing it only buys you less reading
Per-developer leaderboards	Individual accountability	Concealment, which is the most expensive outcome available. A defect board makes people stop filing. A comment board makes people stop thinking

The general principle is Goodhart’s law applied to review: a review metric used as a target stops describing review. The defence is structural, not motivational. Report every metric at team or repository level, on trend rather than absolute value, and always with a paired counter-metric. If a metric cannot be paired, it should not be published.

Does AI code review improve developer productivity?

Direct answer: It moves latency reliably and quality unprovably. Anthropic reports an internal rise from 16% to 54% of pull requests receiving substantive comments, which is coverage, not defect detection, and is self-reported. The only independent converted-to-tests benchmark puts four agents at 41.5% collective issue identification. Measure it with a holdout, not a vendor number. Verified 2026-10-18.

What changes when a machine joins the review is measurement itself, and this is the part most teams miss.

Comment volume stops meaning anything. The moment a bot posts on every pull request, comment count measures the bot’s configuration. Scrutiny density measures the bot’s verbosity. Any dashboard built on comment counts needs a bot-exclusion filter on the day you turn the reviewer on, or your historical trend line breaks permanently.

What you should measure instead, in order:

1. Human scrutiny density, bots excluded. Did humans look harder, or did they look less because the bot was there? This is the single most important question and almost nobody asks it.
2. Rework rate attributable to bot threads. The share of bot comments that caused a code change. This is the tool’s real precision in your repository, and it beats every vendor benchmark because it is measured on your code.
3. Latency to first human review. If the bot posts in four minutes and the human still arrives in nine hours, the bot improved nothing about the bottleneck.
4. Defect escape rate, quarter over quarter. Lagging, noisy, and the only thing that settles the argument.

Now the honest state of the evidence, which is poor.

The independent baseline. c-CRAB (arXiv 2603.23448) converted human review comments into executable tests across 184 pull request instances from 67 repositories, yielding 234 validated review comments. The headline: “Considering the union across all four tools, 97 out of the 234 tests were passed by at least one tool (41.5%).” Per tool: Claude Code 32.1%, Devin 24.8%, PR-Agent 23.1%, Codex 20.1%.

c-CRAB conflict disclosure. Five of six authors are academic (National University of Singapore, Zhejiang University), but Ridwan Shariffdeen lists SonarSource as his affiliation, and the paper carries a disclaimer that its conclusions do not represent SonarSource policy. Sonar sells SonarQube and acquired Gitar in May 2026. Gitar is a paid sponsor of this site. SonarQube is not among the four tools c-CRAB evaluates, which blunts the conflict considerably, but we flag vendor-funded benchmarks everywhere else and so we flag this one.

Also worth knowing: the widely circulated “40 to 70 percent false positive rate” attributed to c-CRAB is not in that paper. No false-positive rate is reported for any reviewed tool. The phrase appears twice and both are conceptual. Do not repeat the miscitation.

The vendor claims, named as such. Anthropic publishes 84% bug detection on pull requests over 1,000 lines at under 1% false positives. Vendor-published and unreplicated, and a sub-1% false-positive rate is extraordinary against every independent measurement available. Entelligence’s own benchmark ranks Entelligence first at 47.2% aggregate F1. Greptile scored Greptile at 82%; Augment’s re-run on the same five repositories scored it 45% F-score, and Augment also rebuilt the answer key (“We expanded and corrected the golden comments by reviewing each PR manually”), so the gap is partly a different answer key rather than purely self-dealing. On Martian’s leaderboard, which is the most credible because Martian sells model routing rather than a reviewer, CodeRabbit sits at 51.2% F1 and 49.2% precision; Qodo then claimed first place at 64.3% and Baz also claimed first. Four self-declared winners, one leaderboard. Full accounting in AI code review benchmarks compared.

The line we will stand behind: no party without a commercial stake has published a false-positive rate, precision figure or noise measurement for any shipping commercial AI code reviewer. SWE-PRBench and CR-Bench do publish false-positive data, but they measure bare models in fixed harnesses, not products you can buy.

So run a holdout. Enable the reviewer on half your repositories for a quarter. Compare human scrutiny density, rework rate and defect escape rate between the halves. It is the only evidence that will survive a budget conversation. Method in how to evaluate an AI code reviewer and the human comparison in AI code review vs manual review.

Best automated code review for reducing reviewer workload

Direct answer: Workload reduction comes from pre-filtering the trivial half of the queue before a human opens the diff, and from a severity floor that stops the tool adding to the queue. CodeRabbit, Greptile and cubic post on pull request open. Qodo Merge is the only one exposing a numeric severity gate. Verified 2026-10-18.

Tool	Workload mechanism	Severity or noise control	Cost shape (verified 2026-10-18)
CodeRabbit	Posts on PR open with a summary plus inline findings; path_instructions narrows per path	reviews.path_filters, reviews.path_instructions. No numeric severity gate	Essentials $24/dev/mo annual, Team $48, Advanced $72. Overage $0.25 per reviewed file. Hourly limits 5/8/10/12 by tier, degrading to 1/hour under fair usage
Greptile	Whole-repo indexing catches cross-file issues a human would need the codebase to find	Natural-language rules; Apex depth as an explicit escalation at 10 credits	Pro $30/seat/mo, 50 credits included, $1 per extra. Credits are per author and do not pool
cubic	Posts on PR open; line allowances attach to seats	Configured per org	No free tier - the live pricing page showed only Team/Pro/Max on 2026-10-18. A previously documented Starter tier with 20 PR reviews/mo per org no longer appears. Team $30, Pro $79, Max $160 per dev/mo yearly, with 40K/80K/160K reviewed lines per dev
Qodo Merge	Inline findings with routing policy	[review_agent].inline_comments_severity_threshold, a severity rank of 1 to 3. Plus comments_location_policy and per-tier comments_routing	Flat team fee from about $30/mo up to 30 users, credits at $0.012 each, pooled across the team, no rollover
Claude Code review	Multi-agent topology with a verification step that attempts to disprove each finding before it is posted, then an aggregator that dedupes and ranks	REVIEW.md from the base branch; three admin-set depth tiers (Quick, Standard, Deep). Nit capping requires the root file on a protected default branch	Claude Team and Enterprise only. Never approves or blocks
GitHub Copilot	Native to the pull request UI; the 300-file and 20,000-line caps were removed 27 Aug 2026	Repository instructions	Included on Copilot Pro ($10), Pro+, Max, Business and Enterprise. Only Copilot Free is excluded. Since 1 June 2026, review on private repos also consumes GitHub Actions minutes

The mechanism that actually reduces workload is the verification step, not the model. Anthropic’s documented topology runs parallel specialist agents (logic, boundary conditions, API misuse, auth flaws, conventions), then a step that tries to disprove each finding, then an aggregator that dedupes and ranks. That shape is designed to reduce the thing that costs reviewers time, which is reading wrong comments. Whether it works is unmeasured by anyone outside Anthropic.

The mechanism that increases workload, and is sold as the opposite, is an unfiltered reviewer on a large or old codebase. If you are pointing one of these at a 15-year-old repository, read AI code review for legacy codebases first: the default configuration produces a wall of technically correct findings and your reviewer workload goes up, not down. Noise scoring method in our noise index methodology and the current evidence in AI code review tools with low false positives.

Best code review tools for reducing time spent on PRs

Direct answer: Time spent on pull requests is mostly waiting, not reviewing. Our data puts 52.7% of a median pull request’s life in the queue before the first human. Routing tools (LinearB, Graphite, Axolo) attack that; AI reviewers attack the 6.7 hours of actual review. Fix routing first. Measured 2026-10-18.

The decomposition decides the purchase:

Where the time goes	Our 2026 measurement	What reduces it
Open to first human review	p50 8.7 hours, p90 212 hours	CODEOWNERS with explicit per-path owners; per-reviewer queue caps; assignment alerts. LinearB and Graphite for visibility, Axolo for Slack-channel-per-PR routing
First review to merge	p50 6.7 hours	Smaller pull requests; an AI reviewer that fixes the obvious before the human arrives; stacked diffs
First approval to merge	p50 1.1 hours	Almost nothing left to win here. Do not optimise it
Author rework between rounds	Not separately measured	Better pull request descriptions, pre-commit hooks, and a bot that catches mechanical issues on push

The ordering is the recommendation. A team buying an AI reviewer to fix a 212-hour p90 has misdiagnosed the problem: the bot posts in minutes, the human still arrives on day nine, and nothing structural changed. Fix assignment, then buy the reviewer. Size is the other lever, and it compounds: there is a 9x spread in median time-to-first-review between sub-50-line and 1,000-plus-line pull requests in our data, so decomposition buys latency directly.

If you are gating on an AI review check in CI, one hard failure mode: GitHub’s merge queue fires a merge_group event separate from pull_request and push. A required review job wired only to pull_request stalls the queue permanently, because the required check is never reported. See what is a merge queue.

Where DORA fits, and what we could not verify about it

Direct answer: DORA’s four keys measure the delivery system, not the review step. Code review sits inside lead time for changes. Change failure rate is the closest DORA proxy for review effectiveness, but it moves for many reasons other than review. Use DORA for the system and the five metrics above for the review step. Verified 2026-10-18.

The four keys are deployment frequency, lead time for changes, change failure rate, and time to restore service. Review shows up inside lead time for changes, and our 52.7%-waiting figure says review latency is often the single largest component of it. That is the honest and useful connection: if your lead time is bad, check the review queue before you touch CI.

What we could not verify, stated plainly: the elite, high, medium and low band thresholds. These are quoted constantly (and in our own DORA metrics explained guide) with specific numbers attached, and we were not able to confirm the current threshold values against a primary DORA or Google Cloud source on 2026-10-18. Treat any specific band number you see, including ours, as unverified until you check it at source. The four keys themselves are not in doubt; the bands are. DORA’s model has since moved to five metrics: mean time to recovery was renamed and redefined as failed deployment recovery time in 2023, and deployment rework rate was added in 2024 because change failure rate had been acting as a proxy for the amount of rework (dora.dev, verified 2026-10-19).

A second caution on DORA and review: change failure rate is a tempting review KPI and a bad one. It moves on deploy tooling, feature flagging, test coverage, infrastructure and luck. Attributing a change in it to review quality requires a holdout, which is the same answer as everywhere else in this article.

How we verified this

Checked on 2026-10-18.

Our own primary measurements, both harvested from the GitHub GraphQL API on 2026-10-18 and published with full method:

• PR size and scrutiny: 2,896 human-authored merged pull requests, 10 major public repositories, 104 bot authors removed, created 17 April to 17 October 2026. Scrutiny density 1.55 under 50 lines falling to 0.085 over 1,000 lines; 52% of merged pull requests over 1,000 lines with zero inline comments; Spearman rho +0.355 for absolute comment counts against size. Method and limits: how big should a pull request be.
• Review latency: 2,189 human-authored merged pull requests, 12 repositories, bot authors and bot reviewers removed, self-reviews excluded. Open to first review p50 8.7h, p75 66h, p90 212h, p95 482h, p99 approximately 1,300h. First review to merge 6.7h at p50; first approval to merge 1.1h. 52.7% median waiting share. 59.2% single-approver, mean 1.08. Method and limits: pull request review time.
• Disclosed limits on both: merged pull requests only, which biases latency downward; Kubernetes and Rust excluded from approver analysis as Prow and bors artifacts; public repositories with volunteer reviewers, so the latency tail is worse than a funded team’s.

Verified against primary sources:

• c-CRAB (arXiv 2603.23448) for 184 pull request instances, 67 repositories, 234 validated review comments and the 41.5% union figure, with the SonarSource affiliation disclosure above.
• SWE-PRBench (arXiv 2603.26130) for 8 frontier models at 15 to 31 percent detection on diff-only review, and the per-model false-positive rates.
• GitHub’s documentation for Copilot code review approvals: “When approvals are enabled, Copilot can submit an approving review that satisfies your repository’s required-approval rule the same way a teammate’s approval would.” Off by default. required_approving_review_count range 0 to 6.
• Google’s public engineering practices guide: “100 lines is usually a reasonable size for a CL, and 1000 lines is usually too large.” There is no 200-line Google rule, and the widely repeated attribution is a misattribution. The 200 to 400 LOC, 60 to 90 minute and 70 to 90 percent figures are real and verbatim on smartbear.com, but they describe a 2006 Cisco Systems program.
• BLS March 2026 private-industry total compensation at $46.60 per hour, used only to argue against single-figure ROI models.

What we could not verify:

• DORA elite, high, medium and low band thresholds. Not confirmed at a primary source on this date. Do not quote specific bands, including from our own guide, without checking.
• Any independent measurement of whether AI code review improves developer productivity. None exists. Anthropic’s 16% to 54% substantive-comment figure is self-reported coverage, not defect detection. Anthropic’s 84% detection at under 1% false positives is vendor-published and unreplicated.
• Public baselines for defect escape rate and rework rate. Neither is computable from public repository data, so we give instrumentation rules and directional ranges rather than numbers we would have to invent.
• Scrutiny density has no prior published baseline we could find. Ours is the first we are aware of. Treat it as a reference point, not a standard.

Recommendations

Criteria: the metric must be computable from data you already have, it must have a stated failure mode, and it must be meaningful at team level without being usable as an individual score.

1. Instrument two metrics this week. Open-to-first-human-review at p50 and p90, and scrutiny density by size band. Both come straight out of the GitHub or GitLab API, and together they tell you whether your review system is slow, shallow, or both.

2. Publish p90, not p50. The median describes the pull requests nobody worries about. Ours is 8.7 hours against a 212-hour p90, and the p90 is where the cost lives.

3. Pair every metric. Latency with scrutiny density. Scrutiny density with rework rate. Merge volume with defect escape rate. An unpaired metric is a gaming invitation.

4. Add a bot-exclusion filter before you turn on an AI reviewer, not after. Otherwise your historical comment and density trends break permanently on day one.

5. Fix routing before buying tooling. 52.7% of a median pull request’s life is spent waiting for a first human. CODEOWNERS and queue caps are free; LinearB and Graphite make the queue visible. An AI reviewer attacks the smaller half of the problem.

6. Run a holdout before you believe any vendor number. Half your repositories, one quarter, comparing human scrutiny density, rework rate and defect escape rate. Every published benchmark in this market except a small handful was funded by a company selling a reviewer, and the independent ones measure bare models rather than products.

7. Never rank individuals. All five metrics turn destructive the moment they become a leaderboard. This is not a cultural preference, it is the predictable consequence of Goodhart’s law on a metric with a cheap gaming strategy.

Related reading: pull request review best practices, code review best practices, how to write a pull request template, what is code coverage, and AI code review for legacy codebases if your repository predates your metrics.