Chapter 10 - Engineering Controls: Make Invalid Work Fail Automatically

Part: III - The Hierarchy Of AI Controls

Thesis

Tests, schemas, type systems, linters, CI, and reducers are not bureaucracy. They are reflexes - and at runtime, the same reflexes are the load-bearing wall of the whole system. A verifier’s authority comes from external rules, state, tests, or oracles, not from the model’s confidence. That is what makes verification stable across model versions, even as the models themselves change constantly.

Key Line

A reflex is a validation step that fires before the system has to think.

What A Reflex Is

A reflex does not deliberate.

When a doctor taps your knee, your leg moves before you decide anything. That movement is the reflex. It does not wait for judgment. It does not depend on attention. It fires because the system is designed to fire it.

Engineering controls work the same way. A failing test does not ask whether you remembered to think about edge cases. It fails. A type error does not wait for a code review to notice the interface drift. It fails at compile time. A schema validator does not trust the model’s confident JSON output. It checks it against the contract and rejects what does not match.

These controls do not get tired. They do not get distracted. They do not forget to run because the deadline is tomorrow. They fire every time the condition is met.

That is what makes them more reliable than any prompt instruction or human review process. They are not better humans. They are mechanisms that close the gap human attention leaves open.

Tests Are Reflexes

A test that passes is not a guarantee. A test that fails is a signal that cannot be ignored.

The value of a test in AI-assisted development is specifically this: when the model generates code that looks correct and is not, the test fails before the code is committed. The model’s confident output does not become a quiet regression. It becomes a loud failure.

This is why “tests as reflexes” is not a metaphor. The test fires automatically. It fires on every relevant change. It does not wait to be run manually. It does not depend on the developer remembering to check the behavior the test covers.

Unit tests are reflexes against logic errors. Integration tests are reflexes against component boundary failures. Regression tests are reflexes against problems that were fixed once and would return silently.

The model can draft all three. Chapter 2 established this. The economic argument is simple: if the test can be drafted cheaply by the model and reviewed by a person, the excuse for having no tests evaporates. Teams that skip tests because writing them is expensive are operating under a constraint that AI partially lifts.

Raise the floor. Write the reflex.

Schema Validation Is A Boundary

A model that produces JSON output is a model that sometimes produces slightly wrong JSON.

Not wrong because it is malfunctioning. Wrong because the output is a continuation of the conversation, not a lookup against a contract. The model does not know that status must be one of "pending", "active", or "closed". It knows the word “status” and the values it has seen in similar contexts. If the session drifted, it might output "in_progress" instead. The value is plausible. It will parse as JSON. It will pass unvalidated into the downstream system and cause a failure hours later, during processing, far from the point where the wrong value was produced.

Schema validation stops this at the boundary.

A JSON Schema, a Zod validator, a Pydantic model, a TypeSpec contract - each is a structural description of what valid output looks like. The model’s output is measured against that description before it reaches any sink. If the output is invalid, the failure is immediate, localized, and legible. The field is wrong. The type is wrong. The value is not in the enumeration.

This is the engineering control equivalent of a type system for data that flows between components. It does not require trusting the model. It requires describing what valid looks like, and then checking.

Teams working with AI-generated output should treat schema validation as mandatory on every route where model output reaches a database, a UI, an API response, or an external service. Not optional. Not aspirational. Required.

Type Systems Are Continuous Contract Checking

A type system is a reflex that runs on every save, every build, every CI run.

When an agent modifies a function signature, the type system immediately surfaces every call site that no longer matches. The engineer does not need to trace the callers. The compiler does. When a model generates code that calls a method that does not exist, the type error appears before the code ships. The invented method cannot hide behind a successful parse.

This is why typed languages and typed tool interfaces reduce AI-assisted development failures. The model generates plausible code. The type system checks plausible against real. The gap between them is the reflex’s job.

Strict types are not pedantry. They are a continuous audit of whether the model’s output matches the actual system.

In practice: use the strictest type settings the codebase can support. Turn on strictNullChecks. Enable exhaustive pattern matching. Use enumerations instead of raw strings where values are finite. Each strictness setting is a category of model hallucination that can no longer reach production silently.

CI Is The Gatekeeper That Does Not Get Tired

A continuous integration pipeline is a gatekeeper.

It runs on every proposed change. It does not have good days and bad days. It does not let something through because it is 5pm on a Friday and the team is tired. It does not have a backlog of other things it is thinking about. It runs the suite, it checks the types, it validates the schema, and it reports the result.

CI is where all the other engineering controls compound. The unit tests run. The type check runs. The lint rules run. The contract tests run. The output is binary: this change passes every reflex the team has installed, or it does not.

This is why CI is the non-negotiable engineering control in AI-assisted development. An agent can produce many changes per session. Without CI, a human reviewer must hold all the engineering controls in their head simultaneously while reading a diff. With CI, the controls run before the human reads anything. The human is asked to review what passed, not to catch everything the controls would have caught.

Branch protection tied to CI is the policy form of this. The branch will not merge until the controls pass. Not “until someone tries to remember to run them.” Until they pass.

Runtime Verifiers Are The Same Reflex

Everything said so far about tests, schemas, and type systems is a build-time story: a reflex that fires when code is written, before it merges. A live AI-native application needs the identical reflex firing on every request, because there is no merge step standing between a runtime proposal and the user who sees its effect.

Runtime verifiers take different shapes than their build-time counterparts, but they do the same job:

• Schema validators check a tool call or state patch against its contract, the same way build-time schema validation checks model-generated JSON - except now it runs on every request instead of every commit.
• Permission checks verify that this caller, in this context, may attempt this action - the runtime equivalent of branch protection.
• Legality engines decide whether a proposed action is valid given current state - a chess rules engine, a game’s turn logic, a workflow’s state machine.
• Deterministic solvers compute the authoritative answer a model may only interpret or explain - ProveCalc’s calculation engine.
• Source-binding gates check that a claim is actually supported by a cited source before the claim may become public - Clio’s provenance check.
• State-transition reducers are both a validator and the sole path into durable state: a proposal that does not pass through the reducer’s checks does not become state, structurally, not by policy.
• Registry lookups resolve a name, an entity, or a reference against an authoritative list rather than trusting the model’s memory of what a valid reference looks like.

None of these depend on the model behaving well. They depend on being built correctly and run on every relevant proposal. That is the entire point: a verifier’s authority is external to the model, so it holds regardless of which model produced the proposal, how confidently, or how the session got there.

Five Kinds Of Validation

Not every check proves the same thing, and conflating them is a common source of false confidence.

Structural validation proves shape: the JSON matches the schema, the types are correct, required fields are present. It proves nothing about whether the content is true, permitted, or wise.

Semantic validation proves meaning against a rule: a date is in the future, a percentage is between 0 and 100, an enum value is one of the declared options. Stronger than structural, still blind to context.

Stateful validation proves consistency with current, authoritative state: this move is legal on this board right now, this balance can cover this withdrawal right now. This is where most of the interesting failures in AI-native systems live, because state changes between the moment context was retrieved and the moment a proposal tries to commit.

Authorization validation proves permission: this actor may take this action on this resource. A structurally perfect, board-legal move proposed by a player whose turn it is not, is still rejected here.

Oracle-backed validation proves quality or correctness against an external authority that is not the primary rules system: Stockfish scoring a legal-but-weak chess move, a second model checking a claim against retrieved sources, a domain expert’s heuristic flagging a plausible-but-unusual pattern. Oracles often produce a graded signal rather than a binary pass or fail.

A proposal can pass one and fail the next. A legal chess move can be a poor one. A well-formed refund request can be for the wrong account. A syntactically valid API call can arrive from a caller who should never be allowed to make it. Treat these as five separate questions, because a system that only asks the first is answering none of the others.

Verifiers Are Model-Independent. They Are Not Infallible.

A verifier’s defining strength is that its behavior does not change when the underlying model changes. Swap GPT for Claude for a fine-tuned open model, and the rules engine still rejects the same illegal moves the same way. That stability is worth protecting, and it is why verification belongs outside the model rather than inside a prompt.

But external does not mean infallible. A schema can drift out of sync with the database it is meant to describe. A permission table can have a stale role mapping. A rules engine can have its own bugs. A solver can have an edge case it handles incorrectly. Verifier coverage - the set of failure modes it actually catches - is never automatically complete just because the verifier is external to the model.

This has a direct consequence: verifiers must themselves be tested, versioned, and audited, the same way the code under test is. A validator with no tests of its own is a control nobody has verified is actually controlling anything.

Verifier Signals

A verifier that runs silently is a verifier nobody is learning from. Track these signals, and read them as diagnostic evidence about the system, not as a scorecard for the model:

• Rejection rate - how often proposals fail validation. Rising rejection rate on stable input usually means specification debt, not sudden model degradation.
• Rejection precision - of the proposals a verifier rejects, how many were actually invalid, versus false rejections of legitimate proposals.
• Rejection recall - of the proposals that were actually invalid, how many the verifier caught, versus ones that slipped through.
• Repair success rate - of proposals sent back for a bounded repair loop, how many come back valid within the allowed attempts.
• Escape rate - how often an invalid proposal reaches its commit boundary anyway. This is Chapter 6’s mixed-output escape rate, generalized to every verifier in the system, and it is the number that matters most.

A system with zero rejections is not necessarily a healthy system. It may mean the model never proposes anything invalid - or it may mean the verifier is not actually checking what it claims to check.

What Happens After Rejection

A rejected proposal needs a defined next step, chosen deliberately per verifier rather than defaulted to whatever is easiest to implement:

• Discard. The proposal is wrong and there is nothing useful to do with it beyond logging that it happened.
• Trace. The rejection is recorded with enough detail to analyze later, even if no immediate action follows.
• Repair. The system returns the specific failure to the model and allows a bounded number of corrected attempts - Chapter 9’s structured failure response, applied here.
• Escalate. The proposal is routed to a human, because the failure mode is ambiguous, high-consequence, or outside what the verifier is confident adjudicating.
• Show the human directly. In some interfaces, the rejected proposal itself - “the model suggested this, and it was rejected because X” - is useful transparency, not just an internal log line.

Choosing “discard” for everything is cheap and throws away diagnostic signal. Choosing “escalate” for everything defeats the purpose of having a verifier at all. The right choice depends on the consequence of the failure and the cost of human attention - the same tradeoff Chapter 33 treats directly for autonomy decisions generally.

Runtime Examples

ProveCalc. The verifier is the deterministic solver itself. The model’s proposal - a problem interpretation, an assumption set - is only trustworthy once the solver, not the model, produces the numeric result. There is no partial credit for a plausible-sounding calculation the solver has not confirmed.

RPG-MCP. The verifier is the rules engine, checking stateful validity (is this move legal on this board, on this turn) before a state-transition reducer will accept it. A structurally valid action schema is necessary and nowhere near sufficient.

LLM-Chess. Two verifiers do different jobs. The legality checker proves stateful validity - is this move even allowed. Stockfish is an oracle - it proves quality, a graded signal about whether a legal move was any good. Confusing the two - treating a legal move as automatically a good one - throws away exactly the distinction this chapter is making.

Clio. The verifier is the source-binding gate: a claim does not reach public state until its citation is checked against the source it claims to represent. This is authorization-adjacent validation for facts instead of actions.

Stagehand. Command validators check every staging directive against current scene state before execution - a structural check (is this a known command), a stateful check (does the referenced entity currently exist), and a domain check (is this transition currently valid) in sequence.

The Labor Economics Argument

Chapter 2 established that attention to detail was always a labor constraint.

Engineering controls are the output of that labor. Writing a test takes time. Setting up schema validation takes time. Configuring strict type settings takes time. Building a CI pipeline takes time.

AI changes the cost of several of these. The model can draft a test matrix from a description of the intended behavior. It can propose a JSON Schema from example outputs. It can write the first version of the contract test. It can produce the Zod validator from the TypeScript interface. It can configure the GitHub Actions workflow from a description of what should run.

The draft is not automatically correct. The team still needs to review, validate, and own the controls. But the cost of the first draft dropped. That matters because engineering controls that do not exist because they were expensive to create are gaps that stay open.

The argument is not “AI will write all your tests for you.” The argument is: the excuse for having no tests, no schema validators, and no CI pipeline is weaker now. Use the leverage.

Every engineering control the team can draft cheaply and validate quickly is a reflex the system gains. Each reflex closes a gap that would otherwise require human attention on every commit.

Raise the floor. The model can help draft the controls. The team’s job is to verify that the controls are true.

Practical Artifact - Engineering Controls Inventory

The first table audits build-time controls: what each one catches, where it runs, and whether AI can help draft it.

Control	What it catches	Where it runs	Required?	Can AI draft it?
Unit tests	Logic errors and regressions in individual functions	Local and CI	Yes	Yes - from behavior description
Integration tests	Boundary failures between components or services	CI (and optionally local)	Yes	Yes - from interface descriptions
Type check (strict)	Interface drift, invented methods, null violations	Local (on save) and CI	Yes	Partially - AI can propose types, human verifies
Schema validation	Invalid model output before it reaches a sink	Runtime (at boundary) and tests	Yes	Yes - from example outputs or TypeScript types
Contract tests	API shape changes that would break callers	CI	Yes, if multi-service	Yes - from OpenAPI spec or interface definitions
Lint rules	Style violations, simple logic errors, forbidden patterns	Local and CI	Yes	Configuration only; AI can propose rule sets
Snapshot / golden file tests	Unexpected changes to stable output (HTML, JSON, SQL)	CI	Recommended	Yes - AI can generate initial snapshots
Branch protection + CI gate	Merges before controls pass	Repository platform	Yes	Not applicable - policy configuration
Regression tests	Recurrence of previously fixed bugs	CI	Yes	Yes - from bug report or reproduction steps
Reproducible build check	Build output differences between environments	CI	Recommended	Partially - AI can propose lockfile and env config

For each row marked “Required?”, treat an empty or disabled state as a gap. Prioritize filling required gaps before extending AI agent capability.

A system with no unit tests, no type checking, and no CI pipeline is a system where every AI-generated change must be caught by human review alone. That is a system running on PPE.

The second table audits runtime verifiers the same way tests audit build-time controls - naming what each one proves and does not prove, and where its blind spots are.

Proposal type	Consequential sink	Validator class	Source of truth	False-accept risk	False-reject risk	Failure behavior	Trace record	Coverage test
Tool call	Executes a capability	Structural + authorization	Permission table	Unauthorized action executes	Legitimate call blocked	Discard, log	Every call, accepted or rejected	Fuzz schema, adversarial auth attempts
Game move	Board state write	Stateful	Rules engine	Illegal move applied	Legal move rejected	Repair (one retry), else discard	Move attempt + verdict	Exhaustive legal-move test suite
Numeric answer	Recorded calculation	Oracle (deterministic solver)	Solver, unit system	N/A - solver is authoritative	Solver rejects valid input format	Return solver error to model	Full derivation, not just result	Known-answer regression set
Public claim	Published content	Authorization + oracle	Source cards, provenance	Unsupported claim published	Supported claim blocked pending review	Escalate to human review	Claim + citation + verdict	Adversarial fabricated-citation set
State patch	Durable state write	Stateful + structural	Reducer, current state snapshot	Patch applied against stale state	Valid patch rejected on race	Reject, request refresh	Patch + pre-state + post-state	Concurrent-write race tests

Reflexes do not tire. Install them first - at every commit boundary the system has, not only the one at Git merge.