IonWarpBest model for docs code review: benchmark (Sep 2026)
Gemini 3.8 Flash has the highest F1 of the 3 models graded on every PR (68.6%) and is also the cheapest within 2 points of it, at $0.08 a review.
| Reviewer / model | F1 | Precision | Recall | $ per review | p50 time | PRs |
|---|---|---|---|---|---|---|
| IonWarp Docs · Gemini 3.8 Flash | 68.6% | 66.7% | 70.6% | $0.08 | 59 sec | 19/19 |
| IonWarp Docs · DeepSeek V4 Pro | 65.2% | 51.7% | 88.2% | $0.06 | 3m 0s | 19/19 |
| IonWarp Docs · Muse Spark 1.3 | 59.5% | 55.0% | 64.7% | $0.09 | 48 sec | 19/19 |
| CodeRabbit | 55.6% | 55.6% | 55.6% | ~$1.50 | 7m 43s | 11/19 |
| Cursor Bugbot | 36.4% | 100.0% | 22.2% | ~$1.00–1.50 | 9m 53s | 11/19 |
| Greptile v4.1 | 28.6% | 40.0% | 22.2% | ~$1.50–2.00 | 5m 23s | 11/19 |
| Qodo v2 | 26.7% | 33.3% | 22.2% | ~$1.67 | 6m 29s | 11/19 |
Rows graded on every PR come first, ranked by Reviewer Score: F1 counts 3×, review time 1× and cost 1×. App prices (~) are list-price estimates; model prices are measured inference cost per review. Provisional labels: IonWarp chose which issues count as Docs Review findings, and no independent party has checked that choice.
19 PRs · snapshot · How we measure