Another AI Reviewed My Protocol. That's Not the Same as Verifying It.

A few days ago I asked Kimi K3, a model I hadn't run past this project before, to look at AAHP, the handoff protocol I've been building in the open. I didn't prime it with my own writing about it, I just pointed it at the repo. What came back was sharp. It correctly picked out the two-axis claim model in the Grounded Reflection Layer as addressing "a real and underappreciated failure mode." It called ADR-017 "an honest but significant admission" that a piece of the protocol's own governance has no enforcement teeth. It didn't just skim the README, it found the seams.

For about thirty seconds I felt good about that. Then I realized I was doing the exact thing the protocol I'd just had reviewed is built to catch.

The trap inside a good review

AAHP's Grounded Reflection Layer describes every claim on two separate axes. The first is status: whether something is verified, assumed, or untested. The second, orthogonal to the first, is provenance: how that status was actually earned. Provenance runs on an ordered scale, weakest to strongest: model_claim, self_reviewed, cross_model_reviewed, source_verified, tool_verified, test_verified, runtime_observed, human_confirmed.

An independent model reading my repo and writing a review of it is, by definition, cross_model_reviewed. And the protocol has a rule for exactly that case, one I wrote before Kimi ever touched the project: cross_model_reviewed maps to status assumed. Never verified. Consensus between models raises robustness. It is not an external anchor.

Kimi's review was accurate on almost every point I could check. That's genuinely useful, it's the kind of second opinion that catches things I've gone blind to from staring at my own spec too long. But useful and verified are different words for a reason. A claim only reaches verified status with something external backing it: passing tests, a build, a type-check, a schema validation, a source I can point to, something a human actually confirmed. Agreement from another model, however sharp, is not that.

So this post does two things. It gives Kimi's read the credit it earned, and it separates that from what I can actually stand behind right now, correctly attributed to the release that shipped it.

Getting the version right (the part Kimi didn't need to fix)

Here's where I nearly created my own example of the same trap. I'd started thinking about this post as "v3.9.0, the release with the Grounded Reflection Layer." That's wrong, and it's worth being specific about why.

The Grounded Reflection Layer, the two-axis model Kimi praised, shipped in v3.3.0, back on July 13th. It's still marked Draft v0.1 in the README today, unchanged since. It hasn't moved forward since it landed: same doctrine, same task-type anchor matrix, same scaffolding via aahp migrate-grounding.

v3.9.0 is a different release entirely, from July 26th. It doesn't touch grounding. It's about something that turned out to be a much harder problem than it looked: how do you know a task's acceptance criteria are actually resolved before you let it close?

v3.9.0, and the report that refused to become a gate

The rule sounds simple. Every task gets one canonical Acceptance criteria section. Each criterion is a checkbox. A box only gets checked with evidence behind it: a commit, a PR, a test run, something real. Before a task can close, every remaining box is either checked, explicitly waived with a stated reason, or handed off to a linked follow-up. aahp criteria is the new command that checks a repo against that rule.

What it is not is a gate. It always exits 0. Whatever it finds, the exit code doesn't move. It also stays completely separate from aahp check, the eight config-driven gates I wrote about in the post about v3.8.0. That release was about building gates that fail loud. This one is about knowing when a gate isn't the right tool at all, and that took three rounds of adversarial review to arrive at. The story of those three rounds is more interesting than the feature itself.

Early drafts shipped this as an opt-in strict mode: warn by default, fail the build if you turned it on. Each review round found real defects in the strict detector and fixed them: ordered lists instead of bullets, indented lists, empty sections, headings written the old way. Fixed, fixed, fixed. And each round also turned up a new document shape that slipped through clean.

The one that ended the argument needed nothing exotic. Just this:

## T-001 Example
### Acceptance criteria
- [x] this one really is done

**Note:** the rest of the criteria follow.

- [ ] NOT DONE AND INVISIBLE
- [ ] ALSO NOT DONE AND INVISIBLE

Run the strict checker against that, with the task marked done in the registry, and it reports "no findings, exit 0." Two real unresolved criteria, invisible. The parser reads a bold line as the end of the section, and everything after it stops existing as far as the check is concerned.

That's ADR-017: recognizing a hand-written acceptance-criteria section in arbitrary Markdown is a heuristic, and a heuristic over an open-ended space of document shapes cannot be made sound. A gate is only worth having because green is supposed to mean safe. Wire an unsound heuristic to an exit code and green stops meaning that, it just means the detector didn't happen to trip this time, and people stop reading the actual document because the build passed. That's worse than no check at all.

So the fix wasn't a fourth round of patches. It was removing the gate. The detection stays, the strict mode is gone entirely, not even as an opt-in, and aahp criteria reports what it finds as an advisory. Nothing it says can block anything. It's built to be read, not trusted blind.

The point, restated

I think that's the right shape for a lot of what currently passes as AI governance. A model reviewing another model's output, a checker parsing hand-written prose, a second opinion from a system you didn't build, all of these produce something worth having. None of them are the same as an anchor.

Kimi's review of AAHP was good work, and I'm not walking any of it back. I'm just filing it where the protocol says it belongs: assumed, cross_model_reviewed, useful and not yet grounded. If I want to move it to verified, I know exactly what that takes, because I wrote the rule myself: something outside the model has to check it. A test. A source. A human.

The repo is at github.com/homeofe/AAHP, the v3.9.0 changelog and ADR-017 are right there in the README if you want the full argument, and the posts that got this project here are the original RFC, the v3 concept post, and the one about v3.6.0 through v3.8.0.