PAI · Cross-Family Review · Empirical Report
Why an all-Claude pipeline stays blind to one class of bug, and how a cross-family panel fixes it. The cheapest reviewers catch the most dangerous ones.
A Claude author and a Claude reviewer share a training distribution, so they share blind spots. A whole class of bug passes both, because both miss it the same way. The cross-family panel routes every branch diff through model families with different training distributions. The bet is simple: a real defect rarely survives all of them at once.
Across 55 review runs and 371 findings, the ledger confirms the bet. It also exposes a structure nobody designed for:
The model that finds the most issues is not the model that finds the most dangerous ones. GPT-5.6-Luna leads on raw volume (140 findings, 3.78/run). Gemini 3.6 Flash runs fewer times and surfaces double the criticals (14 vs 7) plus the most blocking verdicts (12). The two axes point at different models. You cannot collapse the panel into one reviewer, because no member wins on both.
The payoff is concrete. Across these 55 runs the panel surfaced 44 critical, push-blocking defects: correctness bugs, spec-vs-code drift, and ledger-invariant violations. A same-family (Claude) reviewer waves that class through most often, because it shares the author's blind spots. The panel runs before every push as a second opinion and cites each project's own invariants by name. The economics are trivial. The default four-family panel costs about 10¢ per review at open-market token rates (see the rate card below). For a regulated money-path codebase, you get a rounding-error insurance policy against the one bug class an all-Claude pipeline stays blind to.
Note. Claude appears on none of these charts, deliberately. Claude is the orchestrator and author, not an independent checker. Measuring Claude here would grade the code's writer on its own work. The leaderboard tracks the independent witnesses only. More below on why Claude is left off the leaderboard.
Every run appends one record per reviewer: counts of critical, high, and medium findings. Ranked by total volume, the panel splits into a high-yield anchor (Luna), two always-on backends (Grok, Gemini), and an escalation bench (Kimi, Sol) that runs only on the hardest diffs.
Plot each reviewer by the two things a review panel cares about: how many findings it raises per run (breadth) against how many of those are critical per run (depth). A single best model would sit in the top-right corner. None does. Breadth and depth pull apart.
Two of the seven families, Grok (xAI) and Gemini (Google), cost about $0.08 per review combined at market (see the rate card below). Together they raise the majority of the panel's criticals. They are not makeweights. They are the core of the panel. Split every finding by which backend raised it:
| Model (family) | Runs | Crit | High | Med | Total | /run | Blk |
|---|---|---|---|---|---|---|---|
| GPT-5.6-Luna (OpenAI) | 37 | 7 | 74 | 59 | 140 | 3.78 | 7 |
| Grok 4.6 (xAI) | 43 | 11 | 50 | 41 | 102 | 2.37 | 9 |
| Gemini 3.6 Flash (Google) | 27 | 14 | 35 | 27 | 76 | 2.81 | 12 |
| Kimi K3 (Moonshot) | 7 | 3 | 6 | 9 | 18 | 2.57 | 2 |
| Nemotron-3 Ultra (NVIDIA) | 6 | 2 | 5 | 8 | 15 | 2.50 | 2 |
| GPT-5.6-Sol (OpenAI) | 3 | 1 | 5 | 1 | 7 | 2.33 | 1 |
| DeepSeek V3.2 | 16 | 2 | 0 | 0 | 2 | 0.13 | 2 |
Why route code past other model families at all? The argument is not "more reviewers is better." It is a first-principles claim about where blind spots come from.
A reviewer detects a defect only if that defect falls inside its visible set: the classes of bug its training distribution makes it sensitive to. Author and reviewer each carry a visible set. A bug is caught if it lands in the union of the visible sets pointed at it.
When author and reviewer share a training distribution (a Claude writing code, a Claude reviewing it), their visible sets are highly correlated, and so are their blind spots. A defect sitting in the shared blind spot passes both, every time, because both miss it the same way. A stronger single reviewer does not fix this. The problem is correlation, not quality. The same principle drives RAID arrays to mix disk batches and aviation to use dissimilar redundancy.
The tempting move is to replace the panel with one better reviewer. The data refutes this directly. The strongest single volume-catcher (Luna) finds half the criticals-per-run that Gemini does. Reviewer strength is not the binding constraint. Blind-spot correlation is. Two independent, weaker reviewers beat one strong reviewer whose blind spots you already share.
To maximize the union of visible sets under a cost budget, the design is clear. Maximize family diversity first (independent distributions). Add the expensive frontier families only where the extra defect-catch justifies the spend. Two genuinely independent families (Grok, Gemini) cost about 8¢ a run combined at market, so near-total distribution diversity costs almost nothing. That is the shape the cost split shows.
The ledger's own numbers make the argument hard to vary. Breadth and depth live in different families, and the corner where a single best reviewer would sit is empty. You cannot compress the panel to one model without losing an axis. The panel is not redundancy. The panel is coverage. That is the whole framework in one sentence.
Claude wrote the code and ran the first review. Claude also orchestrates this panel and adjudicates what comes back. Claude is the one intelligence in the loop that is not an independent witness, so these charts leave it off by design.
The framework exists to escape a single failure mode. An author and a reviewer that share a training distribution share their blind spots, so a class of bug passes both. Claude is the author. A Claude reviewer would then get graded on catching the very bugs it tends to miss. Its findings would not be independent evidence. They would be the same distribution voting twice. Counting them would inflate the panel's apparent coverage with correlated noise.
The division of labour is deliberate. Claude sits upstream. Claude authors the diff, runs the first-pass same-family review, loads the repo's invariants, spawns the panel, and adjudicates the union of findings. The cross-family panel is the independent second opinion. The leaderboard tracks the checkers, never the thing checked. This is also why you cannot build the panel from Claude sub-agents wearing other vendors' names. Independence has to be real, not costumed. The next section shows why.
The panel is a self-activating review skill: a defined contract plus one shell engine, run from inside any git repo. Intent triggers it: "cross-family review", "second opinion", "have another family look at this". The pre-push and pre-PR moment triggers it too. It runs after the same-family Claude review and complements it. It never replaces it.
origin/dev → origin/main → main, or an explicit base).{block, summary} verdict in --json mode. block=true if any reviewer flags a critical (union, not consensus).The price of the panel is the market rate: what each family charges per token on the open market. Live OpenRouter list prices, per 1M tokens (fetched 2026-08-21):
| Reviewer (family) | OpenRouter slug | Input $/M | Output $/M | Market $/run† |
|---|---|---|---|---|
| Grok 4.6 (xAI) | x-ai/grok-4.6 | 2.00 | 6.00 | 0.068 |
| Gemini 3.7 Flash (Google) | google/gemini-3.7-flash | 0.38 | 1.88 | 0.015 |
| GPT-5.6-Luna (OpenAI) | openai/gpt-5.6-luna | 0.20 | 1.20 | 0.009 |
| DeepSeek V3.2 | deepseek/deepseek-v3.2 | 0.27 | 0.40 | 0.008 |
| GPT-5.6-Sol (OpenAI) | openai/gpt-5.6-sol | 2.50 | 15.00 | 0.108 |
| Kimi K3 (Moonshot) | moonshotai/kimi-k3 | 3.00 | 15.00 | 0.120 |
Market $/run is what the whole panel costs at list price for every reviewer.
| Profile | Roster | Market $/run | When |
|---|---|---|---|
| cheap | grok · gemini | ~$0.08 | near-zero smoke pass |
| standard | grok · luna · deepseek · gemini | ~$0.10 | routine second opinion (default) |
| deep | + sol | ~$0.21 | correctness-critical: ledger / auth / migrations |
| max | + sol · kimi | ~$0.33 | genuine architecture decisions only |
The single most consequential architectural decision: reviewers are CLI subprocesses, not Claude Agent sub-agents. This choice makes "cross-family" real instead of cosmetic.
A Claude Agent sub-agent, say a "GrokResearcher" persona, runs on the underlying Claude model. It describes xAI, but it is still Claude underneath: same weights, same training distribution, same blind spots. Dressing a sub-agent in another vendor's name buys zero independence. The only genuinely cross-family path is to shell out to the real binaries and let a different company's model read the diff.
grok (xAI) and agy (Google Gemini) run through their own authenticated CLIs: two independent families invoked directly. These are the core of the cost split.codex. Adding a model is a flag, not a patch.--pr mode. Fetches the PR head fresh into a detached throwaway git worktree and reviews the newest pushed commit, recording against the canonical repo name. Your working tree stays untouched.openai/gpt-5.6-luna, etc.) before name normalization. That is a separate 11-finding tranche, not folded into the leaderboard above. Normalizing reviewer names at write time closes this.If you find these things interesting and would like to work with us at Belong, reach out. https://belong.us