Skip to content
IonWarpTry for free
SOC 2 code review

Add SOC 2 Review
to your audited pull requests.

SOC 2 doesn't explicitly require code review. An approved pull request is the usual CC8.1 proof, but it can't show what the change broke.

atlantis#2436
Make required `atlantis/apply` status check work with `mergeable` using `--gh-allow-mergeable-bypass-apply`
server/events/vcs/github_client.goatlantis #2436
IonWarp caught it 3 other models missed it
IonWarpbotcommented on line 346

P1[CC6] Bypass ignores required checks whose check suite is not completed · SOC 2 Review

GetCombinedStatusMinusApply only inspects check runs that live inside a check suite with `status == "completed"` (line 346). A required check that is still queued or in-progress has a non-completed suite, so it is never examined and the function falls through to `return true, nil` (line 376). When the flag is on and the PR is approved, `PullIsMergeable` therefore reports the PR mergeable even though a required (non-apply) check has not finished, letting apply proceed with an unmet required check.

HighKnown issue in the SOC 2 Review benchmark · caught by DeepSeek V4.1 FlashSee the graded case

What SOC 2 expects from code review

SOC 2 code review needs no particular tool, and the review itself is not explicitly required. The change-management criterion, CC8.1, asks that changes to software are authorized, tested and approved before they are implemented,[1] and the pull request is the usual evidence: a required reviewer approves it, CI tests it, and the merge ties the change to what shipped.

SOC 2 Review adds an automated review to that record. It flags the control gaps a change introduces (a privileged action with no audit record, a CI gate switched off, a secret in a log, encryption turned off) and names the Trust Services criterion each one breaks. On 20 benchmark pull requests with 15 known control gaps, DeepSeek V4.1 Flash, the model it runs, found 13; GLM 5.3 Flash scored the same F1 (66.7%) in 34 seconds for about a cent a review.[2] It produces review evidence; it does not make you compliant or replace the human approval your controls require.

Control gaps a single pull request can introduce

  • An approved pull request can still break a control

    All 15 benchmark pull requests with a known control gap were merged with the gap in the final commit, and 13 carry an approval from someone other than the author.[3] They were chosen for their defects, so don't read this as a rate. An approval proves who signed off, not what the change did to your controls.

  • A required check that stopped counting

    Atlantis #2436 let apply run past required checks still queued or running. On the SOC 2 benchmark DeepSeek V4.1 Flash flagged it; GLM 5.3 Flash, DeepSeek V4 Pro and Muse Spark 1.3 did not.[4]

    SOC 2 ReviewOn Max

  • A decrypted secret returned by an API

    Grafana #130148's new SSO settings store returned the decrypted OAuth client secret in API responses, where the old store redacted it. DeepSeek V4.1 Flash and GLM 5.3 Flash flagged it; DeepSeek V4 Pro, GPT 5.6 Luna Pro and Muse Spark 1.3 did not.[5]

    SOC 2 ReviewOn Max

How an automated review and a human approval become change-management evidence

  1. IonWarp reviews the pull request

    SOC 2 Review posts findings on the pull request, each naming the Trust Services criterion it affects. It joins when the diff touches auth, roles, admin actions, audit logging, CI workflows, secrets or data deletion. For your vendor review, how IonWarp handles your code.

    SOC 2 ReviewOn Max

  2. Findings are resolved on the record

    Each finding is fixed or declined with a reason, and the findings ledger on the pull request shows which.

  3. A person approves, and the merge is traceable

    Your required GitHub reviewer, set with GitHub branch protection rules, approves the change, and the merge ties the change, its review and its approval to the commit you deployed.

SOC 2 Review on the benchmark

ModelF1PrecisionRecall$ per reviewMedian time
DeepSeek V4.1 FlashThe model it runs66.7%54.2%86.7%$0.045m 56s
GLM 5.3 FlashA failed review retries on it · best on this benchmark66.7%61.1%73.3%$0.0134 sec

Why IonWarp runs DeepSeek V4.1 Flash: in a trial of our specialist reviewers on production pull requests on 23 September 2026 it filed about as many findings per review as GLM 5.3 Flash at high effort (0.93 against 1.02) for a third of the cost ($0.039 against $0.125), and 3 of its 64 findings were refuted.

Measured on the same 20 open-source pull requests, with known issues. See the benchmark

IonWarp caught 4 issues other models missed

Real IonWarp comments on open-source pull requests, graded against the benchmark's answer key.

Missed byGPT 5.6 Luna Pro 2DeepSeek V4.1 Flash 0DeepSeek V4 Pro 2Muse Spark 1.3 3

2 of 4 · See the benchmark

pkg/registry/apis/iam/sso/mtsettings_store.gografana #130148
IonWarp caught it 3 other models missed it
IonWarpbotcommented on line 261

P0[C1] Secret-bearing SSO setting values are returned verbatim in API responses · SOC 2 Review

When a provider blob contains a client_secret or private key, rowsToSSOSetting copies each raw row Value into Spec.Settings, which Get/Create/Update return to the caller. Combined with the permissive authorizer, any authenticated identity can read decrypted IdP secrets over the k8s API; no masking or secret redaction is applied on this response path.

HighKnown issue in the SOC 2 Review benchmark · caught by GLM 5.3 FlashSee the graded case

GLM 5.3 Flash is its best model on this benchmark; SOC 2 Review runs DeepSeek V4.1 Flash unless you pick another.

Pricing

Simple pricing for your whole team.

One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.

Starter

$0 /mo

Try AI PR review.

  • 2 reviewers
    • Code Review
    • Security Review
  • 3 seats included
  • 15,000 credits to start, then 5,000/mo
  • About 70 reviews to start, then about 25 a month
  • Unlimited repos
Get Started

Max

$149 /mo

Scale AI PR review.

  • +9 reviewers
    • UX Review
    • API Performance
    • Analytics Review
    • SEO Review
    • Changelog
    • CI/CD Performance
    • Plan Review
    • Repeat-Failure Learnings
    • SOC 2 Review
  • 10 seats included
  • +75,000 credits/mo
  • About 360 reviews a month
  • +$29/mo per added seat
  • Unlimited repos
  • Priority support
Choose Max
Enterprise

Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA

Contact us
FAQ

Frequently asked questions

Does SOC 2 require code review or a specific tool?

Not explicitly, and no tool is prescribed. CC8.1 asks that changes be authorized, designed, developed, configured, documented, tested, approved and implemented. Teams choose the controls that meet it, and pull request review with a required approval is a common one.

Does IonWarp make us SOC 2 compliant?

No. IonWarp produces review evidence on each pull request. Compliance depends on your controls as a whole and on your auditor's assessment.

Can an automated reviewer replace the human approval?

It should not. Keep a required human reviewer in your branch protection; IonWarp's findings inform that approval and stay on the pull request as part of the record.

Which plan includes SOC 2 Review?

SOC 2 Review runs on Max ($149/mo for the workspace with 10 seats included, with a 7-day trial). Security Review runs on Starter, Pro and Max.

Sources

  1. 2017 Trust Services Criteria (With Revised Points of Focus – 2022)AICPA · read
  2. The SOC 2 review benchmark: 20 pull requests, 15 known control gaps (release of 2026-09-24)IonWarp benchmark · read
  3. Pull request reviews API, read for the benchmark's 15 pull requests with a known gapGitHub · read
  4. runatlantis/atlantis #2436 (required checks), graded SOC 2 findingsIonWarp benchmark · read
  5. grafana/grafana #130148 (SSO settings), graded SOC 2 findingsIonWarp benchmark · read