Code review comment examples: real comments, useful and noisy
A useful code review comment names the line, the input that breaks it, what happens, and the fix. The comments below are real: IonWarp's Code Review agent, on DeepSeek V4.1 Flash, posted them on open-source pull requests in our benchmark.
On 50 pull requests it found 77 of the 158 known issues, and 78 of its comments matched none.[1] A comment that matches no known issue is not proof it is wrong, but it is the kind a reader learns to skip.
Updated
Three real comments: two that found a bug, one that matched nothing
“Unhealthy flusher process is never killed for spawn-context processes”
Sentry #93824, flusher.py line 254. Matched a known issue. It names the process type that makes the new isinstance check always false, what follows (a hung process keeps running beside its replacement), and the fix. Qodo, Greptile, Bugbot and CodeRabbit did not post it.[2]
“Non-atomic retry counter increment loses updates across overlapping runs”
Cal.com #14943, scheduleSMSReminders.ts line 184. Matched a known issue. It says when it breaks (two cron runs overlap), what happens (an increment is lost and the same SMS is scheduled twice) and the fix: increment in the database. Bugbot, Greptile and Qodo did not post it.[3]
“Hard-backpressure check issues one cluster INFO query per flusher process”
Sentry #93824. Matched no known issue. It applies only when two settings are changed from their defaults, as the comment itself notes, and it changes no decision, only the number of queries.[2]
What the useful comments have in common
The line and the trigger
The file, the line, and the input or event that breaks it: two cron runs that overlap, a process started with the spawn context.
What happens
What a user or an operator sees: a lost count, a duplicate SMS, a hung process that is never killed.
The fix
One change the author can make: increment the counter in the database, check the process's real base type. Reviewers' shorthand is decoded in PTAL, nit and LGTM.
Sources
- IonWarp benchmark, The code review benchmark: 50 pull requests, 158 known issues (release of 2026-10-06), read 2026-10-12.
- IonWarp benchmark, getsentry/sentry #93824, graded findings, read 2026-10-03.
- IonWarp benchmark, calcom/cal.com #14943 (retry counter), graded findings, read 2026-10-04.
