A model reviewing every diff catches real things. Done naively it also produces so much noise that people stop reading it — which is worse than no reviewer.

1. Review the diff, with context

Send the changed lines plus enough surrounding code to understand them. A diff alone lacks the context to judge whether something is wrong.

code-reviewer.txt
Build a review step that runs on each pull request. Repo: <GitHub /
GitLab>.

- Fetch the diff plus surrounding context for changed files.
- Run separate focused passes rather than one general review:
  correctness, security, and error handling. One call each.
- Each finding: file, line, what's wrong, why it matters, and a
  confidence level.
- Post as inline comments, but only findings above a confidence
  threshold. Say what threshold and why.
- Skip generated files, lockfiles and anything over <n> lines changed.

Cap the spend per pull request and say what happens when it's hit.

2. Focused passes beat one general review

"Review this code" produces style opinions. "Find places where user input reaches a query unescaped" produces findings. Run three or four narrow passes and the signal is dramatically better.

3. Verify before posting

The step that makes this usable. Take each finding and ask a second call to argue against it — is this actually a problem in this context? Drop the ones that don't survive.

This removes a large share of findings, and what remains is worth reading.

4. Be quiet by default

Post nothing when there's nothing to say. A reviewer that comments "looks good, consider adding tests" on every pull request trains people to ignore it.

Aim for a low volume of high-value comments. If it's producing more than a few per change, tighten the passes.

5. Never block

Advisory only, at least to start. A model-driven check that blocks merges will block a correct change eventually, and the resulting frustration ends the experiment.

Track how many findings people act on. If it's under a third, the reviewer is too noisy and you should narrow the passes rather than adding more.