IonWarpStarter
Try AI PR review.
- 2 reviewers
- Code Review
- Security Review
- 3 seats included
- 15,000 credits to start, then 5,000/mo
- Unlimited repos
Security code review
Security code review: what to check, and what AI reviewers catch
A security code review, or secure code review, reads a change for the ways it can be abused: who can reach it, what input it trusts and what it exposes. OWASP's Code Review Guide is the long version;[1] this is the short one, built on real bugs.
We ran 11 AI reviewers on 13 open-source pull requests with 18 known security issues. The best found 11. One issue was caught by all 11, and three by none.[2] AI review is a fast first pass, not a replacement for a person reading the permission logic. The set is small and its answer key is provisional, so we print the table and do not rank on it.
What to check, with real bugs and how many of 11 reviewers caught them
Input that reaches a query: 5 of 11
A Discourse admin controller built SQL by interpolating host names into the string. It should bind them as parameters.[3]
URLs your server fetches: 6 of 11
A Discourse job called open() on a URL from site settings, so whoever sets it can make the server fetch internal addresses.[4]
Input that reaches a page: 4 of 11
The same pull request wrote the Referer header into JavaScript without escaping it, which is cross-site scripting.[4]
Allow lists and block lists: 10 and 2 of 11
A case-sensitive lookup let User@Example.com past a block on user@example.com, and a regex with no boundary let evil.example.com past an entry for example.com.[5]
One-time tokens under load: 3 of 11
Cal.com checked 2FA backup codes in memory and then wrote them back, so two parallel logins could spend the same code.[6]
Login flows: 4 of 11
A Sentry GitHub OAuth flow used a static signature as its state value, not a random one per request.[7]
Response headers: 3 of 11
Discourse set X-Frame-Options to ALLOWALL and trusted the Referer header, which any client can fake, so clickjacking protection was off.[4]
Permission logic: 0 of 11
Grafana trusted cached grants but rechecked cached denials, so a stale grant could reach a revoked resource,[8] and a Keycloak check looked up a resource by the wrong owner, ignoring per-client permissions.[9]
How to run a security code review
Start where trust changes
List what the pull request adds: routes, jobs, webhooks, and any value from a user, a header or a setting. Read those slowly.
Follow each input to where it lands
A query, a shell, a URL fetch, a page. Check it is bound, validated or escaped there, not only at the edge.
Read permission changes by hand
Two of the three issues no reviewer caught were permission logic: seeing them takes knowing how the system names and caches its resources.
Let tools do the pattern work
A SAST tool and secret scanning catch known patterns. An AI reviewer such as Security Review reads the diff as a first pass; how it scored on real pull requests.
Security ReviewOn Starter, Pro and Max
The apps and IonWarp on the same 13 pull requests
18 known security issues. A provisional, unranked table: disputed labels remain open, and each app's general review is scored against a security key. The best model we ran, GLM 5.3 Flash, found 10 with 2 unmatched, at about a cent a review.[2] IonWarp's column is the model its Security Review runs by default.
Simple pricing for your whole team.
One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.
Pro
Most popularGet AI PR review right from day 1.
- +5 reviewers
- DevOps Review
- Docs Review
- AI Architecture
- Log & Errors
- Cost Review
- 5 seats included
- +25,000 credits/mo
- +$29/mo per added seat
- Unlimited repos
Max
Scale AI PR review.
- +9 reviewers
- UX Review
- API Performance
- Analytics Review
- SEO Review
- Changelog
- CI/CD Performance
- Plan Review
- Repeat-Failure Learnings
- SOC 2 Review
- 10 seats included
- +75,000 credits/mo
- +$29/mo per added seat
- Unlimited repos
- Priority support
Enterprise
Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA
Frequently asked questions
What is a security code review?
Reading source code changes for ways they can be abused: injection, broken access control, exposed secrets, unsafe fetches and the like. It is also called a secure code review, and it can be done by a person, a static analysis tool, an AI reviewer or all three.
What should a security code review check?
Input that reaches a query, a shell or a page; URLs the server fetches; allow lists and block lists; tokens and login flows; response headers; permission checks; and secrets in code and logs.
How is a security code review different from SAST?
A SAST tool matches known patterns across a codebase. A security code review reads what a change means, including logic no pattern describes, such as who may call a new route. Most teams run both.
Can AI do a security code review?
As a first pass. On 13 pull requests with 18 known security issues, the best of 11 AI reviewers found 11, and three issues were found by none, two of them permission logic. Keep a person on auth and access changes.
Which AI reviewer finds the most security bugs?
On our provisional benchmark the Qodo app found the most, 11 of 18, with 1 unmatched finding. The best model, GLM 5.3 Flash, found 10 with 2 unmatched. The set is small and its answer key is still disputed, so we print the table and do not rank on it.
Sources
- Code Review Guide
- The security review benchmark: 13 pull requests, 18 known issues (release of 2026-09-24)
- discourse #10 (embeddable hosts), graded security review findings
- discourse #4 (embed URL handling), graded findings
- discourse #3 (email domain allowlist), graded security findings
- calcom/cal.com #10600 (2FA backup codes), graded security findings
- getsentry/sentry #67876 (OAuth state), graded security findings
- grafana/grafana #103633 (authz caching), graded security review findings
- keycloak/keycloak #36880 (client permissions), graded security review findings