IonWarpOn general code review, yes: it was the best model we ran, with an F1 of 56.6% on 50 pull requests, behind only the Qodo app. On security, SOC 2, CI/CD and logs, cheaper models scored higher.
Guide
Muse Spark code review: Muse Spark 1.3 benchmarks on real pull requests
Muse Spark 1.3 is the best model we have run on general code review. On 50 open-source pull requests with 158 known issues, it found 79 with 42 false alarms, an F1 of 56.6%, for 42 cents a review.[1] Only the Qodo app scored higher, 58.5%. We ran it on eight of IonWarp's nine review types (how the benchmarks work).
IonWarp's Code Review still runs DeepSeek v4.1 Flash by default: it scored 46.7% and costs 12 cents a review, under a third of Muse Spark's price.[1] Muse Spark is what a failed Code Review or API Performance review retries on. On most specialist reviews, cheaper models scored higher.
Where Muse Spark 1.3 leads, and where it does not
Code review: the best model, second to one app
Ahead of DeepSeek V4 Pro (51.4%) and every other model we ran.[1] In Keycloak #40940, getSubGroupsCount() could return null when a cached group was deleted at the same moment, though its contract says it never does. Muse Spark flagged it; Bugbot, CodeRabbit and Greptile missed it.[2]
Code ReviewOn Starter, Pro and Max
API performance: the best model, behind three apps
56.0% on 18 pull requests, in 42 seconds for about 6 cents. Greptile (76.9%), Bugbot (71.4%) and Qodo (66.7%) scored higher.[3]
API PerformanceOn Max
Docs: third of three models
59.5% on 19 pull requests, behind Gemini 3.8 Flash (68.6%) and DeepSeek V4 Pro (65.2%).[4]
Docs ReviewOn Pro and Max
Security, SOC 2, CI/CD and logs: behind cheaper models
Security 44.4% at 26 cents a review, against 66.7% for GLM 5.3 Flash at about a cent.[5] SOC 2 55.2% and CI/CD 44.4%, last of five models on both;[6][7] logs 43.6%, third of five.[8]
Starter is free for 3 seats. Pro is $49 a month with 5 seats, and Max is $149 a month with 10 seats. Compare plans
Frequently asked questions
Price. On the code benchmark it costs 42 cents a review against 12 for DeepSeek v4.1 Flash, which IonWarp runs for Code Review. A failed Code Review review retries on Muse Spark.
Muse Spark found more known issues, 79 to 64, at 42 cents a review against 16. DeepSeek V4 Pro was more precise: 70% of its findings matched a known issue, against 65% for Muse Spark.
On code review its median was 3 minutes 18 seconds a review. On API performance and docs it took under a minute.
Sources
- The code review benchmark: 50 pull requests, 158 known issues (release of 2026-09-24) — IonWarp benchmark, read 2026-10-03.
- keycloak/keycloak #40940, graded code review findings — IonWarp benchmark, read 2026-10-04.
- The API performance review benchmark: 18 pull requests, 13 known issues (release of 2026-09-24) — IonWarp benchmark, read 2026-10-04.
- The docs review benchmark: 19 pull requests, 17 known issues (release of 2026-09-22) — IonWarp benchmark, read 2026-10-03.
- The security review benchmark: 13 pull requests, 18 known issues (release of 2026-09-24) — IonWarp benchmark, read 2026-10-03.
- The SOC 2 review benchmark: 20 pull requests, 15 known control gaps (release of 2026-09-24) — IonWarp benchmark, read 2026-10-03.
- The CI/CD benchmark: 18 pull requests, 13 known pipeline defects (release of 2026-09-20) — IonWarp benchmark, read 2026-10-03.
- The log and error review benchmark: 17 pull requests, 31 known issues (release of 2026-09-24) — IonWarp benchmark, read 2026-10-04.
Get a review on your next pull request
Install IonWarp on GitHub. Starter is free for 3 people, with 15,000 credits to start.
Try for free