Chapter 5 - Verify Before Commit

Part: II - Trust, But Verify

Thesis

Verification must happen before state changes, deployment, publication, or trust transfer.

Key Line

Hallucination is inevitable. Silent commit is optional.

Commit Is Bigger Than Git

When most developers hear “commit,” they think of git commit.

That is too small.

A commit is any moment when output becomes accepted state. It is the point at which something generated, proposed, or asserted crosses from being a candidate into being real.

Git commit is one example. But the concept applies everywhere a system accepts input as truth.

Clicking “Apply migration” is a commit. Pressing “Publish” is a commit. Deploying to production is a commit. Inserting a row is a commit. Accepting an agent’s tool call result is a commit. Adding a claim to a public document is a commit. Updating stored credentials is a commit. Signing off on a security assumption is a commit.

Every one of these moments has a before and an after. Before, the output is a proposal. After, it is state. The gap between before and after is where verification must happen. Not after. Not eventually. Before.

The failure to understand this is how AI slop enters systems that nobody intended to be sloppy.

Development Commits And Runtime Commits

Two different clocks produce two different kinds of commit.

A development commit is generated code, tests, schemas, migrations, documentation, or configuration becoming repository or deployed software. A human, or a pipeline a human configured, decides what merges. There is usually a diff, a review window, and time to look before the commit happens.

A runtime commit is generated output becoming database state, an executed tool action, a published claim, a legal game move, a rendered event, or other user-visible truth - produced live, on a machine clock, often with no reviewer in the loop unless the architecture puts one there.

The boundaries share a shape and differ in tempo. Several of the examples below are development commits: a migration reviewed on a pull request, a deployment configuration merged before release. Others are runtime commits: a tool call executed mid-session, user-visible state written on the spot. The discipline is identical either way, but the runtime cases have no reviewer standing by to catch a mistake at the moment it matters - which is exactly why their verification gate has to be built into the system rather than practiced by a person. Chapter 13 develops this distinction into the full Propose / Validate / Commit loop used across the rest of the book.

Generation-Level vs. Commit-Level Hallucination

There are two distinct problems here, and conflating them leads to both the wrong fears and the wrong solutions.

Generation-level hallucination is when the model emits something false. The model names an API that does not exist. It proposes a schema that conflicts with the actual database. It cites a paper that was never published. It writes a test that passes when it should fail. This is normal. It happens constantly. It is not the thing to be most afraid of.

Commit-level hallucination is when the system accepts something false as state.

The model can hallucinate freely in the proposal lane. That is acceptable. The model cannot hallucinate into the production database, the public knowledge base, the deployed codebase, or the trust model - not because hallucination stops there, but because those are the places where false state has real consequences that do not automatically reverse.

The distinction matters because the correct response to each is different.

Generation-level hallucination is managed by expecting it and designing systems that catch it before output crosses a commit boundary. You do not try to prevent the model from hallucinating. You prevent hallucinations from becoming state.

Commit-level hallucination is managed by enforcing verification gates that every proposal must pass before it is accepted as real.

The key line says it plainly: hallucination is inevitable. Silent commit is optional.

The hallucination you cannot prevent. The gate you can choose to build.

Verification Must Match The Property

Passing a gate proves one property. It does not prove the others.

A JSON payload can be perfectly well-formed and still request something the caller is not allowed to do. The schema validator proves structure. It says nothing about authorization - that is a different question, checked by a different component, against a different source of truth: the permission table, not the parser.

A chess move can be completely legal and still be a poor move. The rules engine proves legality. It says nothing about quality. Stockfish, or an equivalent oracle, checks a different property, and Chapter 26 treats that separation directly.

A calculation can use a correctly parsed formula and the wrong assumption. The parser proves the expression is well-formed. The solver - ProveCalc’s deterministic engine, not the model - proves the number, run against the stated units and assumptions. Neither check proves the assumption was the right one to make; that is a product-intent question, and it may still need a human. Chapter 24 covers ProveCalc’s full allocation.

No single validator proves every property a proposal needs. Structure, authorization, state legality, computation, factual support, and product intent are six different claims. Each one needs a check built for that claim. A system that runs one gate and calls the proposal “verified” has usually verified exactly one of these and assumed the rest.

The Code Commit

The most familiar commit. A model proposes a diff. The developer reads it. It looks right.

Silent commit: the developer adds the changes, sees no compilation errors, and pushes.

What it costs: the change introduces a subtle race condition that only appears under concurrent load. The bug lives in production for three weeks before anyone correlates it with the commit. Nobody knows what the original code was supposed to do, so the fix requires archaeology.

The verification gate: automated tests, type checking, and at minimum, a diff review that asks “what assumption did the model make that I have not verified?” A passing test suite does not guarantee correctness. It does mean the change has been subjected to something more rigorous than appearance.

The Database Commit

A model proposes a migration. The developer reviews the SQL. It looks reasonable.

Silent commit: the migration runs against production without a backup, without a dry run, without a rollback plan.

What it costs: the migration drops a column that was referenced by an application query path the developer did not know about. Service degrades. The rollback requires restoring from the last backup. Four hours of data are gone.

The verification gate: dry run on a staging copy, explicit rollback plan, backup confirmation before running, and a check against all references to the columns being altered. The model can propose the migration. It cannot know every query path in the application. The developer must close that gap before the migration touches real data.

The Public Content Commit

A model drafts a public-facing article, briefing, or announcement. It sounds authoritative and polished.

Silent commit: the content is published without checking whether the factual claims in it are true.

What it costs: the article states that a specific regulation took effect in a particular year. The year is wrong. The piece goes out to ten thousand readers. Corrections require public notice. Trust degrades.

The verification gate: every factual claim in public content needs a source. Not a citation the model invented - a source a human checked. Claims that cannot be sourced should be labeled as uncertain or removed. The polish of the prose is not evidence of the accuracy of the claims.

The Production Deployment

A model proposes infrastructure changes or a new deployment configuration. The configuration looks clean.

Silent commit: the deployment goes to production without staging validation, without a rollback path, without a canary release.

What it costs: the configuration introduces an environment variable mismatch that causes the service to fail on cold start. The failure is not caught until traffic arrives. Recovery takes forty minutes and a late-night rollback.

The verification gate: staging environment runs first, metrics confirm baseline behavior, deployment is staged or canary before full rollout, and a named rollback procedure exists before the production deploy begins.

The Documentation Commit

A model updates the README or API docs. The generated text matches what the model thinks the code does.

Silent commit: the docs are committed without checking whether they accurately describe current behavior.

What it costs: a new team member follows the documented setup steps and cannot get the service running. The steps describe a configuration that was changed six weeks ago. The developer who changed it did not update the docs. The new member spends half a day on a false start.

The verification gate: documentation claims must be checked against the actual code, the actual commands, and the actual behavior. If the README documents a command, run the command and confirm it works. If the API docs document a response shape, confirm the response shape against the current schema. Generated docs that have not been verified against behavior are not documentation. They are plausible fiction.

The Security Assumption Commit

A model reviews an authentication flow and confirms it looks correct. The review seems thorough.

Silent commit: the security review is accepted as sufficient without independent verification of the assumptions the model made.

What it costs: the flow is vulnerable to a token replay attack the model did not consider because it was not in the prompt. The vulnerability is exploited three months later. The “security review” provides false confidence that makes the team less likely to investigate.

The verification gate: security assumptions must be validated by threat modeling, not by whether a review sounds thorough. What are the assets being protected? What are the trust boundaries? What capabilities does an attacker have? What is the failure mode if an assumption is wrong? The model can propose answers. Those answers are not verified until they have been tested against real threat scenarios.

The Agent Trust Transfer

A model calls a tool and reports the result. The calling system accepts the report.

Silent commit: the result is added to application state without confirming that the tool call succeeded in the way the model claims.

What it costs: the model reports that a file was written. The file was not written because a permission error occurred and the model did not surface the failure correctly. The downstream task assumes the file exists. Silent failure propagates through the pipeline.

The verification gate: tool calls have receipts. The system does not accept a model’s report that a tool call succeeded - it checks the side effect directly. If a file was supposed to be written, the file is checked. If an API call was supposed to update a record, the record is read back. Agent-to-agent trust transfer requires the same skepticism as human-to-system trust: verify the state, not the claim.

After The Gate Says No

A verification gate is not finished when it says no. Rejection is an event, not a dead end, and different failures deserve different handling.

Discard fits a proposal that is cheap and easy to regenerate - a phrasing that missed the tone, a comment that added nothing. Nothing is lost by dropping it.

Repair fits a small, mechanical failure - a missing field, a name that does not match the registry. Feeding the specific validator error back to the model often produces a passing proposal on the next attempt; Chapter 20 turns this into a debugging pattern.

Retry without new information is not a fix. Resending the same prompt against the same gap in the specification just resamples the same mistake in different words.

Escalate fits a property the system cannot yet check by itself: an ambiguous authorization case, a claim with no available source, a product-intent question no validator was built to answer. Escalation routes the proposal to a human with the rejection reason attached, rather than committing it or discarding it.

Trace happens every time, regardless of which of the above also applies. A rejected proposal is evidence about where the specification or the validator is still weak. Losing that record loses the one signal that would have shown the gap.

Building To The Key Line

The model will hallucinate. This is not a limitation that better models will eliminate. It is a structural property of systems that generate probable continuations. Even a model that hallucinates less will hallucinate. Even a model that sounds more confident will sometimes be wrong with that same confidence.

You cannot prevent the generation.

You can build the gate.

Hallucination is inevitable. Silent commit is optional.

Every system that accepts AI output as state is making a choice about where the gate lives. The question is not whether to have a gate. The question is whether the gate is explicit and enforced, or implicit and skipped.

An explicit gate is a test that runs. A validation command that must pass. A dry run that must succeed. A source that must be checked. A reviewer who must be satisfied. A side effect that must be confirmed.

An implicit gate is a feeling that the output looked right.

Feelings do not catch generation-level hallucinations before they become commit-level ones.

This is the exact point where deterministic software stops being optional scaffolding and becomes the constraint structure the model operates inside. Every gate in this chapter - a test suite, a dry run, a source check, a permission table, a side-effect confirmation - is deterministic or externally authoritative precisely so its answer does not depend on how convincing the proposal sounded. Verify before commit is not a review habit. It is the architecture.

Practical Artifact

Use this Commit Boundary Map to identify where verification must happen in your workflow. Complete one row for each stage where AI-generated output becomes accepted state.

Workflow stage	Commit type	Boundary	What becomes state	Silent commit risk	Verification gate	Gate owner
AI proposes code diff	Code commit	Development	Git history, deployed behavior	Bugs ship undetected	Tests, type check, lint, diff review	Developer
AI proposes DB migration	Database commit	Development	Schema, stored data	Data loss, broken queries	Dry run, backup check, rollback plan	Developer + DBA
AI drafts public content	Content commit	Runtime	Public knowledge, trust	False claims published	Source check, human editorial review	Editor/owner
AI generates deployment config	Deployment commit	Development	Production environment	Outage, regression	Staging run, canary, rollback plan	DevOps/owner
AI updates documentation	Docs commit	Development	Team knowledge, onboarding	False instructions	Behavioral verification against code	Doc owner
AI produces security review	Security commit	Development	Trust model, threat posture	False confidence	Threat model validation, independent review	Security owner
Agent reports tool call result	Trust transfer	Runtime	Application state	Silent pipeline failure	Side-effect confirmation, not just claim	System design
AI generates user-visible state	UI commit	Runtime	User experience, user trust	Wrong information displayed	Data validation, acceptance test	Product owner

Fill in the “Gate owner” column for your team. If no name goes in that cell, the gate does not exist.

Add rows for every workflow stage where AI output becomes state in your system.

If a stage has no verification gate, that is a silent commit risk. Name the gate or name the risk explicitly.

Mark the boundary honestly. A development commit usually has a human-paced review window behind it. A runtime commit usually does not - which is why runtime rows need the strongest gates, built in advance, not the most trust.

The goal is not a full table. The goal is a system where every path from generation to state passes through something.