Open benchmark · CC BY 4.0
Which model reviews code best
Every review app and model we measure, on the same real pull requests with known issues. 9 lanes, 186 pull requests, 4 apps and 9 models. The cases and the verdicts are public, so you can check our numbers, including where we lose.
9:41

Benchmark · Ranked
Code Review
50 real pull requests, 158 known issues
ReviewerScoreF1
- 1Muse Spark 1.3Model71.256.6%
- 2Qodo v2Review app64.758.5%
- 3DeepSeek V4 ProModel48.751.4%
- 4DeepSeek V4.1 FlashCode Review runs this41.049.2%
- 5GLM 5.3 FlashModel40.842.8%
- 6Greptile v4.1Review app39.951.1%
- 7Cursor BugbotReview app36.251.4%
- 8GPT 5.6 Luna ProModel31.841.6%
- 9CodeRabbitReview app3.142.1%
| Model / App | Reviewer Score | F1 | Latency | $/run |
|---|---|---|---|---|
| IonWarp · Muse Spark 1.3 | 71.2 | 56.6% | 3m 18s | $0.424 |
| Qodo v2 | 64.7 | 58.5% | 5m 40s | ~$1.67 |
| IonWarp · DeepSeek V4 Pro | 48.7 | 51.4% | 5m 59s | $0.165 |
| IonWarp · DeepSeek V4.1 FlashRuns on your PRs | 41.0 | 49.2% | 6m 56s | $0.102 |
| IonWarp · GLM 5.3 Flash | 40.8 | 42.8% | 2m 15s | $0.018 |
| Greptile v4.1 | 39.9 | 51.1% | 4m 58s | ~$1.75 |
| Cursor Bugbot | 36.2 | 51.4% | 7m 52s | ~$1.25 |
| IonWarp · GPT 5.6 Luna Pro | 31.8 | 41.6% | 1m 47s | $0.116 |
| CodeRabbit | 3.1 | 42.1% | 7m 34s | ~$1.50 |
Reviewer Score = (3 × F1 + time + $/run) ÷ 5, each part 0 to 100 within this table. IonWarp rows cost the model’s inference; ~$ is each app’s list price for 5 seats at 100 reviews a month. Pinned Martian Core reproduction. Release 2026-10-12, CC BY 4.0.
Qodo v2 leads on F1 at 58.5%. IonWarp’s Code Review runs DeepSeek V4.1 Flash: 49.2% at $0.102 a review. Every row and every PR →
We also publish a comparison of ten AI code review tools by price and Git host at aicodereviewtools.dev.
Code Review
IonWarp inferenceApp price estimatesF1 on the same 50 open-source pull requests for every reviewer
Lanes
One lane per kind of review
Each agent is measured on pull requests with known issues of its own kind. Only Code Review is ranked; the others are provisional until their case sets grow.

Security Review
Qodo v2 leads, F1 73.3%. Security Review on DeepSeek V4.1 Flash: 62.5%.

AI Architecture
CodeRabbit leads, F1 72.7%. AI Architecture on DeepSeek V4.1 Flash: 47.1%.

Log & Errors
Qodo v2 leads, F1 72.7%. Log & Errors on DeepSeek V4.1 Flash: 59.0%.

Docs Review
IonWarp · Gemini 3.8 Flash leads, F1 68.6%. DeepSeek V4.1 Flash not measured yet.

API Performance
IonWarp · Muse Spark 1.3 leads, F1 56.0%. API Performance on DeepSeek V4.1 Flash: 53.3%.

SOC 2 Review
IonWarp · GLM 5.3 Flash leads, F1 66.7%. SOC 2 Review on DeepSeek V4.1 Flash: 66.7%.

CI/CD Performance
IonWarp · DeepSeek V4 Pro leads, F1 72.7%. CI/CD Performance on DeepSeek V4.1 Flash: 59.5%.

Cost Review
IonWarp · DeepSeek V4.1 Flash leads, F1 57.1%. Not one of the eight agents.
API
The same data, as JSON
Read-only, open to any origin, cached at the edge. Cite it as “IonWarp AI code review benchmark, CC BY 4.0”.
curl https://ionwarp.com/benchmarks/v1/families
curl https://ionwarp.com/benchmarks/v1/reviewers/ionwarp-code-review/metrics- /benchmarks/v1/families: every lane, with the model IonWarp runs on it.
- /benchmarks/v1/reviewers/<lane>/metrics: one lane’s full table, PR by PR.
- /benchmarks/v1/openapi.json: the schema.
FAQ
About the benchmark
What is the best AI code review tool?
On the code review lane (50 pull requests, 158 known issues), Qodo v2 has the best F1 at 58.5%. IonWarp’s Code Review, on DeepSeek V4.1 Flash, scores 49.2% at $0.102 a review in model cost. The full table is below.
How is an AI code reviewer scored?
A judge model matches each review comment to the pull request’s known issues. Precision is the share of comments that match one, recall the share of known issues caught, and F1 balances the two. The Reviewer Score adds speed and cost: (3 × F1 + time + cost) ÷ 5.
Can I check the numbers?
Yes. Every row, pull request and verdict is on GitHub under CC BY 4.0, and the same data is served at /benchmarks/v1 as JSON.
Why is only Code Review ranked?
Its 50 pull requests and labels come from a public corpus. The specialist lanes are smaller, some labels are ours, and they stay provisional until two raters agree on them.
Read next
Read next
Put the leaders on your next PR.
Try for free14 days free, no card. Then Pro is $49 a month per workspace with 5 seats, or Max $149 with 10.