Try for free

Ranked · release 2026-10-12

Code Review benchmark

50 PRs, 158 known issues. Qodo v2 leads at F1 58.5%. Code Review on DeepSeek V4.1 Flash: F1 49.2% at $0.102 a review.

About this lane
  • Pinned Martian Core reproduction.
  • Judge: Martian Core, Claude Opus 4.5 judge.
Code Review: how each reviewer scoresRanked
Code Review: how each reviewer scores
Model / AppReviewer ScoreF1Latency$/run
IonWarp · Muse Spark 1.371.256.6%3m 18s$0.424
Qodo v264.758.5%5m 40s~$1.67
IonWarp · DeepSeek V4 Pro48.751.4%5m 59s$0.165
IonWarp · DeepSeek V4.1 FlashRuns on your PRs41.049.2%6m 56s$0.102
IonWarp · GLM 5.3 Flash40.842.8%2m 15s$0.018
Greptile v4.139.951.1%4m 58s~$1.75
Cursor Bugbot36.251.4%7m 52s~$1.25
IonWarp · GPT 5.6 Luna Pro31.841.6%1m 47s$0.116
CodeRabbit3.142.1%7m 34s~$1.50

Reviewer Score = (3 × F1 + time + $/run) ÷ 5, each part 0 to 100 within this table. IonWarp rows cost the model’s inference; ~$ is each app’s list price for 5 seats at 100 reviews a month. Pinned Martian Core reproduction. Release 2026-10-12, CC BY 4.0.

Code Review

IonWarp inferenceApp price estimatesF1 on the same 50 open-source pull requests for every reviewer

40%50%60%$0.03$0.10$0.30$1.00Cost per review · USD · log scaleQodo v2Muse Spark 1.3DeepSeek V4 ProCursor BugbotGreptile v4.1DeepSeek V4.1 FlashGLM 5.3 FlashCodeRabbitGPT 5.6 Luna Pro
Precision, recall and the counts behind F1
Code Review: precision, recall and the counts behind F1
Model / appEffortPrecisionRecallCaughtUnmatchedMissedPRsTime p90Measured
IonWarp · Muse Spark 1.3medium65.3%50.0%79427950 / 507m 31s2026-09-08 – 2026-09-11
Qodo v2—55.4%62.0%98796050 / 508m 10s2026-02-27 – 2026-02-27
IonWarp · DeepSeek V4 Promedium70.3%40.5%64279450 / 5013m 19s2026-09-11 – 2026-09-13
IonWarp · DeepSeek V4.1 Flashmedium49.7%48.7%77788150 / 508m 15s2026-10-05 – 2026-10-06
IonWarp · GLM 5.3 Flashlow50.0%37.3%59599950 / 509m 52s2026-09-08 – 2026-09-11
Greptile v4.1—50.9%51.3%81787750 / 507m 09s2026-04-07 – 2026-04-07
Cursor Bugbot—56.9%46.8%74568450 / 5011m 41s2026-01-28 – 2026-01-28
IonWarp · GPT 5.6 Luna Promedium73.0%29.1%461711250 / 502m 44s2026-09-11 – 2026-09-13
CodeRabbit—32.6%59.5%941946450 / 5012m 10s2026-01-23 – 2026-04-05
Every pull request: known issues caught by each reviewer50 PRs

A cell is how many of the PR’s known issues that reviewer caught; links open its review. – means the reviewer was not run on that PR. On a phone this shows the first 3 of 9 reviewers; a wider screen shows them all.

Put Code Review on your next PR.

Try for free

14 days free, no card. Then Pro is $49 a month per workspace with 5 seats, or Max $149 with 10.