Starter
Try AI PR review.
- 2 reviewers
- Code Review
- Security Review
- 3 seats included
- 15,000 credits to start, then 5,000/mo
- About 70 reviews to start, then about 25 a month
- Unlimited repos
SOC 2 doesn't explicitly require code review. An approved pull request is the usual CC8.1 proof, but it can't show what the change broke.
HighKnown issue in the SOC 2 Review benchmark · caught by DeepSeek V4.1 FlashSee the graded case
SOC 2 code review needs no particular tool, and the review itself is not explicitly required. The change-management criterion, CC8.1, asks that changes to software are authorized, tested and approved before they are implemented,[1] and the pull request is the usual evidence: a required reviewer approves it, CI tests it, and the merge ties the change to what shipped.
SOC 2 Review adds an automated review to that record. It flags the control gaps a change introduces (a privileged action with no audit record, a CI gate switched off, a secret in a log, encryption turned off) and names the Trust Services criterion each one breaks. On 20 benchmark pull requests with 15 known control gaps, DeepSeek V4.1 Flash, the model it runs, found 13; GLM 5.3 Flash scored the same F1 (66.7%) in 34 seconds for about a cent a review.[2] It produces review evidence; it does not make you compliant or replace the human approval your controls require.
All 15 benchmark pull requests with a known control gap were merged with the gap in the final commit, and 13 carry an approval from someone other than the author.[3] They were chosen for their defects, so don't read this as a rate. An approval proves who signed off, not what the change did to your controls.
Atlantis #2436 let apply run past required checks still queued or running. On the SOC 2 benchmark DeepSeek V4.1 Flash flagged it; GLM 5.3 Flash, DeepSeek V4 Pro and Muse Spark 1.3 did not.[4]
SOC 2 ReviewOn Max
Grafana #130148's new SSO settings store returned the decrypted OAuth client secret in API responses, where the old store redacted it. DeepSeek V4.1 Flash and GLM 5.3 Flash flagged it; DeepSeek V4 Pro, GPT 5.6 Luna Pro and Muse Spark 1.3 did not.[5]
SOC 2 ReviewOn Max
SOC 2 Review posts findings on the pull request, each naming the Trust Services criterion it affects. It joins when the diff touches auth, roles, admin actions, audit logging, CI workflows, secrets or data deletion. For your vendor review, how IonWarp handles your code.
SOC 2 ReviewOn Max
Each finding is fixed or declined with a reason, and the findings ledger on the pull request shows which.
Your required GitHub reviewer, set with GitHub branch protection rules, approves the change, and the merge ties the change, its review and its approval to the commit you deployed.
| Model | F1 | Precision | Recall | $ per review | Median time |
|---|---|---|---|---|---|
| DeepSeek V4.1 FlashThe model it runs | 66.7% | 54.2% | 86.7% | $0.04 | 5m 56s |
| GLM 5.3 FlashA failed review retries on it · best on this benchmark | 66.7% | 61.1% | 73.3% | $0.01 | 34 sec |
Why IonWarp runs DeepSeek V4.1 Flash: in a trial of our specialist reviewers on production pull requests on 23 September 2026 it filed about as many findings per review as GLM 5.3 Flash at high effort (0.93 against 1.02) for a third of the cost ($0.039 against $0.125), and 3 of its 64 findings were refuted.
Measured on the same 20 open-source pull requests, with known issues. See the benchmark
Real IonWarp comments on open-source pull requests, graded against the benchmark's answer key.
Missed byGPT 5.6 Luna Pro 2DeepSeek V4.1 Flash 0DeepSeek V4 Pro 2Muse Spark 1.3 3
2 of 4 · See the benchmark
P0[C1] Secret-bearing SSO setting values are returned verbatim in API responses · SOC 2 Review
When a provider blob contains a client_secret or private key, rowsToSSOSetting copies each raw row Value into Spec.Settings, which Get/Create/Update return to the caller. Combined with the permissive authorizer, any authenticated identity can read decrypted IdP secrets over the k8s API; no masking or secret redaction is applied on this response path.
HighKnown issue in the SOC 2 Review benchmark · caught by GLM 5.3 FlashSee the graded case
P1[CC7] Audit record truncated when DisablePostgres is set: traits and actor_name never reach the log entry · SOC 2 Review
When GOTRUE_AUDIT_LOG_DISABLE_POSTGRES is enabled, NewAuditLogEntry returns at `if config.DisablePostgres { return nil }` (line 120-122) BEFORE `actor_name` enrichment (line 124-126) and the `traits` assignment (line 128-130). observability.LogEntrySetFields at line 116-118 has already emitted only the base payload (actor_id, action, log_type), so with Postgres writes off, the sole remaining audit record for privileged actions like adminUserDelete or UnenrollFactor lacks the target identifiers (user_id, factor_id, session_id) and actor name — the record is incomplete, not just relocated.
if config.DisablePostgres && false { // move below traits enrichmentMediumKnown issue in the SOC 2 Review benchmark · caught by GLM 5.3 FlashSee the graded case
GLM 5.3 Flash is its best model on this benchmark; SOC 2 Review runs DeepSeek V4.1 Flash unless you pick another.
One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.
Try AI PR review.
Get AI PR review right from day 1.
Scale AI PR review.
Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA
Not explicitly, and no tool is prescribed. CC8.1 asks that changes be authorized, designed, developed, configured, documented, tested, approved and implemented. Teams choose the controls that meet it, and pull request review with a required approval is a common one.
No. IonWarp produces review evidence on each pull request. Compliance depends on your controls as a whole and on your auditor's assessment.
It should not. Keep a required human reviewer in your branch protection; IonWarp's findings inform that approval and stay on the pull request as part of the record.
SOC 2 Review runs on Max ($149/mo for the workspace with 10 seats included, with a 7-day trial). Security Review runs on Starter, Pro and Max.
P1[CC6] Bypass ignores required checks whose check suite is not completed · SOC 2 Review
GetCombinedStatusMinusApply only inspects check runs that live inside a check suite with `status == "completed"` (line 346). A required check that is still queued or in-progress has a non-completed suite, so it is never examined and the function falls through to `return true, nil` (line 376). When the flag is on and the PR is approved, `PullIsMergeable` therefore reports the PR mergeable even though a required (non-apply) check has not finished, letting apply proceed with an unmet required check.