IonWarpBest model for SOC 2 code review: benchmark (Sep 2026)
DeepSeek V4.1 Flash has the highest F1 of the 5 models graded on every PR (66.7%, $0.04 a review); GLM 5.3 Flash is the cheapest within 2 points of it, at $0.01 a review.
| Reviewer / model | F1 | Precision | Recall | $ per review | p50 time | PRs |
|---|---|---|---|---|---|---|
| IonWarp SOC 2 · GLM 5.3 Flash | 66.7% | 61.1% | 73.3% | $0.01 | 34 sec | 20/20 |
| IonWarp SOC 2 · GPT 5.6 Luna Pro | 64.5% | 62.5% | 66.7% | $0.07 | 37 sec | 20/20 |
| IonWarp SOC 2 · DeepSeek V4.1 Flash | 66.7% | 54.2% | 86.7% | $0.04 | 5m 56s | 20/20 |
| IonWarp SOC 2 · DeepSeek V4 Pro | 56.3% | 52.9% | 60.0% | $0.07 | 2m 21s | 20/20 |
| IonWarp SOC 2 · Muse Spark 1.3 | 55.2% | 57.1% | 53.3% | $0.11 | 48 sec | 20/20 |
| CodeRabbit | N/A | N/A | N/A | ~$1.50 | N/A | N/A |
| Cursor Bugbot | N/A | N/A | N/A | ~$1.00–1.50 | N/A | N/A |
| Greptile v4.1 | N/A | N/A | N/A | ~$1.50–2.00 | N/A | N/A |
| Qodo v2 | N/A | N/A | N/A | ~$1.67 | N/A | N/A |
Rows graded on every PR come first, ranked by Reviewer Score: F1 counts 3×, review time 1× and cost 1×. App prices (~) are list-price estimates; model prices are measured inference cost per review. Provisional labels: IonWarp chose which issues count as SOC 2 Review findings, and no independent party has checked that choice.
20 PRs · snapshot · How we measure