GPT as a Read-Only Reviewer for Claude Code: Adversarial Review With Pi

When Claude reviews code it just wrote, it mostly catches things it would have thought of anyway. A model from another company, on its first real run, caught a regression Claude had written and then missed in its own review.

The setup: I took Pi, a terminal coding agent, connected it to GPT models through an API, and wrapped it as a read-only reviewer that Claude Code can call.

Two conclusions came out of it. Permission rules are not a sandbox: for real read-only, don't hand out write-capable tools in the first place. And adversarial review only converges once the reviewer knows the scope and how the last round's findings were handled.

I wanted that second opinion with one requirement: the reviewer can't modify files. Whether and how to change anything is up to me and Claude.

To use it, I say "have pi review this" or type /pi-review in Claude Code. Claude hands Pi the uncommitted changes, a branch, a PR, or a plan document.

When the findings come back, Claude opens the code and checks each one, marks it agree, disagree, or can't confirm, and reports to me.

Only two Pi packages

• earendil-works/pi-review: the official /review, with findings graded P0 to P3
• @gotgenes/pi-permission-system: allow / ask / deny rules for tools, bash commands, and paths

Two I looked at but didn't install. pi-web-access uses Exa search by default, which sends code snippets to an outside service. pi-simplify edits files directly.

I also considered wrapping Pi as an MCP server for Claude Code to call. The only pi-mcp-server I found is no longer maintained, and the common approach in the community is to just shell out.

Pi automatically reads the project's CLAUDE.md and AGENTS.md, so reviews apply the same project rules without me spelling them out again.

Permission rules are not a sandbox

The rules: edit and write are always denied. Bash defaults to ask, with only read-type commands plus git diff/log/show/blame and gh pr diff/view allowed.

.env, .ssh, .aws, and Pi's own auth.json are blocked even from reading. In a test repo I confirmed the write tool was hidden and touch was blocked.

But that's a rule layer, not a sandbox.

In interactive mode, an allowed read command plus a redirect, like cat a > b, can still write a file.

The first time I used it for real, I started pi by hand without --tools. Pi had bash, actually ran rg and git diff, and the permission rules were the only thing in the way.

Since then, the skill always starts Pi with -na --tools read,grep,find,ls, so there's no bash at all. From Pi's CLI docs:

Default enabled tools are read, bash, edit, and write, unless defaultTools changes them. --tools with plain names replaces the whole selection, so name every tool you want.

Leave out --tools and bash is in the default set. Pass it, and only the named tools are left.

The diff is written to a file first and Pi reads it with read. When I asked it directly which tools it had, it listed only read, grep, find, and ls.

The permission rules drop back to a second layer of insurance. They block most cases but only work as a backup. If you want read-only, take the write tools out of the launch arguments.

There are two ways to run it. Inside herdr (a terminal multiplexer for coding agents), it opens a Pi panel on the right, so I can see what it's reading, and the session sticks around for multiple rounds. The other is headless pi -p, with the diff fed in on stdin.

Three rounds to converge

The prompt to Pi is adversarial. It has to verify whether the design the author claims actually holds up, and attack spots like edge-case inputs, rolling deploys, and transaction boundaries.

If nothing is wrong, the first line has to say "No findings that need fixing", so Claude can tell from the first line whether another round is needed.

Only when I explicitly say to fix things does it go into the fix loop. Code fixes need new tests plus mutation checks, where you break the code on purpose and confirm a test fails.

The first real run took three rounds. Round one caught a regression Claude had introduced: on write, it filled in fields with old values it had read earlier, which would overwrite values someone else had just changed.

Claude hadn't seen this in its self-review.

Converging wasn't that smooth. At first every round turned up problems in existing mechanisms that had nothing to do with this change.

It settled only after I added three requirements to the prompt. Mark issues that already exist on the base branch as "pre-existing". Report only issues within the scope of this change. Include how each finding from the previous round was handled.

The result: round one had 2 High and 1 Medium, round two had 1 needing confirmation and 2 Medium, and round three came back with "No findings that need fixing".

Without the scope and the previous round's dispositions, adversarial review keeps picking at pre-existing issues and never converges.

Review history stays in my notes

Code comments written while fixing findings only explain why the code is written the way it is now. They don't say "raised by pi" and don't describe the old version.

The review history goes only into my own notes, never into docs that get committed.

A model from another company really does see things self-review misses. The regression in round one is the example.