Refactoring With AI
The safer the test harness, the more aggressive the refactor can be.
Chapter 21 - Refactoring With AI
Part: V - AI-Assisted Development Workflows
Thesis
AI-assisted refactoring has a blast radius proportional to the diff size. Refactors must be reviewed for behavioral change, not just syntactic change.
Key Line
The safer the test harness, the more aggressive the refactor can be.
What Refactoring Is And Is Not
A refactor preserves observable behavior while changing internal structure. That definition is precise and the precision matters.
A change that modifies behavior is not a refactor. It is a feature, a bug fix, or a bug introduction wearing a refactor’s label. The distinction is not semantic - it determines what verification is required. Behavioral changes need acceptance criteria. Refactors need behavioral equivalence.
The model does not track this boundary on its own. Ask it to refactor a function and it will produce cleaner code. It will also, silently, make choices: rename variables to follow a different convention, change a default value to match a pattern it learned, alter a guard clause to what seems more correct, handle an edge case differently than the original. Each choice looks like an improvement. Each choice may change what the code does.
AI-assisted refactoring is not dangerous because the model is careless. It is dangerous because the model optimizes for clean code, and clean code is not the same as equivalent code.
The AI’s Refactoring Problem
The model generates code that looks refactored. The problem is invisible without testing.
When a person manually refactors, they make explicit, tracked decisions. They rename a variable and know exactly why. They extract a function and know what it contains. The changes are incremental and intentional. The person’s mental model updates with each change.
When the model refactors, it transforms the entire function in one generation pass. The output is the result of thousands of small decisions made in parallel, none of them explicitly tracked. The result looks clean. The behavioral delta from the original is not surfaced anywhere.
This is not a reason to avoid AI-assisted refactoring. It is a reason to verify it systematically. The model is fast and good at structural cleanup. The verification step that checks for behavioral equivalence is the human’s job.
The Behavior-Preserving Constraint
Before any refactor, enumerate the observable behaviors the code must preserve.
“Observable behaviors” means: what does this code return for given inputs, what side effects does it produce, what exceptions does it raise and when, what does it do when inputs are at boundary values. Write these down. Write them as tests if they are not already tests.
The list does not have to be exhaustive. It has to be honest. List the behaviors you know about and the edge cases that have historically mattered. The refactor is only complete when all of those behaviors still hold.
This list also defines the scope of verification. A refactor of a pure utility function with well-defined inputs and outputs is verifiable with unit tests. A refactor of a system integration point requires integration tests. The required verification level is a function of the observable behaviors, not of the code’s appearance.
Refactors Can Move The Trust Boundary
A refactor can pass every test and still make the system less safe, because tests check behavior, and a trust boundary does not always show up in behavior.
A trust boundary is the line where the system stops trusting an input and starts requiring it to be checked: a permission check that runs before a delete, a schema validation that runs before a tool call executes, the order in which an authorization check and a domain check run. None of these show up as a return value. All of them show up in an incident if a refactor moves them.
The failure looks clean in the diff. A function that validated input and then checked permission gets “simplified” into a function that checks permission and then validates input, because the two checks looked logically independent. In the common case both orders behave the same. In the case that matters - a malformed request that should have failed validation before permission was ever consulted - the system now logs a permission denial instead of a validation failure, and a check meant to run against clean input now runs against an unvalidated one.
Six things must survive a refactor unchanged, even when every line around them is rewritten:
- Schemas. The contract a value must satisfy does not loosen because the function that checks it got shorter.
- Validator order. If validation ran before authorization, it still runs before authorization.
- Authorization rules. Who is allowed to do what does not change because the enforcing code moved to a different file or abstraction.
- State ownership. The component that owned a piece of state before the refactor still owns it after.
- Trace semantics. What gets logged, when, and with what detail does not shrink because a log call looked redundant.
- Rollback behavior. If the pre-refactor code could be safely reversed, the post-refactor code can too.
None of these are about what a function returns. They are about what it is allowed to do, to whom, and in what order. A test suite that only checks return values does not see any of them move. Before accepting any AI-proposed refactor that touches a schema, a validator, an authorization check, a state write, a trace call, or a rollback path, name what that code protects and confirm the protection is unchanged.
Test Coverage As Prerequisite
You cannot safely refactor code that has no tests. This is not a guideline. It is a structural constraint.
The refactor will produce a green CI run, but the CI cannot tell you whether behavior changed in untested cases. Green tests are evidence of correctness for the tested cases. They say nothing about the rest. A model-generated refactor on untested code gives you cleaner-looking code with an unknown behavioral delta. That is a liability dressed as progress.
The path forward is not to skip the tests and hope. Add tests first. Focus on the observable behaviors that matter most: the happy path, the known edge cases, and any behavior that has produced a bug in the past. These tests define the behavioral contract. Then run the refactor against the tests.
If adding tests before refactoring reveals that the existing code’s behavior is not what you assumed, that is not a problem with the process. That is the process finding a latent bug before the refactor hides it deeper.
Scope Discipline
A refactor that touches 500 lines is not a refactor. It is a rewrite.
Rewrites lose behavioral context. When every function is touched, every variable is renamed, and every control flow path is restructured, the connection between the new code and the known behaviors of the old code is severed. The tests that pass on the new code prove the new code works. They do not prove it is equivalent to the old code, because no one has read both closely enough to confirm the mapping.
Break large refactors into small, independently verifiable steps. Each step has one objective: rename these variables, extract this function, eliminate this duplication, move this module boundary. Each step is verified before the next begins. The test suite must pass at every intermediate point.
The constraint is operational: any intermediate state must be releasable. If a refactor requires a multi-day intermediate state that cannot be shipped, the scope is too large. Break it further.
Boundary-Diff Review
Add one question to every refactor review, separate from “does this do the same thing”: for each piece of code that touches truth, authorization, or state, what component owned that truth before the refactor, and what component owns it after?
Answer it as a before/after pair, not a single check.
| Before the refactor | After the refactor | Same owner? |
|---|---|---|
| Permission table checked in the route handler | Permission table checked in the service layer | Yes - moved, not removed |
| Order total computed by the pricing engine | Order total computed inline by a “simplified” helper | No - truth moved to a function nobody reviews as an oracle |
| Refund amount validated against the ledger | Refund amount validated against the model’s summary of the ledger | No - a deterministic source of truth was replaced with a model’s paraphrase |
The first row is a safe refactor: the boundary moved, but the same component still owns the same truth, checking the same rule. The second and third rows are not refactors. They are undisclosed feature changes wearing a refactor’s name, because the source of truth for a fact changed. A model-proposed refactor is a proposal like any other model output; it does not get to claim contract equivalence for itself. Prove it by naming the owner on both sides and confirming it is the same one.
Do Not Refactor Determinism Into Judgment
One specific boundary move deserves its own warning: replacing explicit deterministic logic with a model call, in the name of shorter code.
A twenty-line function that validates an address against a fixed set of rules is not improved by asking a model to “just check if the address looks valid.” The twenty lines are slower to read and faster to trust. The model call is faster to read and cannot be trusted the same way, because its judgment varies by input, by model version, and by nothing visible in a diff. A refactor whose main achievement is replacing deterministic logic with a prompt has not reduced complexity. It has moved complexity into a component that cannot be unit tested the way the code it replaced could be, and it has relocated a source of truth from the codebase into the model’s weights.
This is not an argument against ever calling a model from application code. It is an argument against doing so under the banner of “refactoring,” where the reviewer’s guard is down because the diff is billed as behavior-preserving cleanup.
Reviewing For Behavioral Change
When reviewing a refactor diff, the question is not “does this look cleaner?” The question is “does this do the same thing?”
The places behavioral changes hide in refactored code are consistent:
Renamed variables. A variable renamed from timeout_ms to timeout may now be used in a context where the unit is assumed. The rename was cosmetic. The bug is semantic.
Changed defaults. A function parameter that previously defaulted to false now defaults to true because the model followed a different pattern. Callers that did not pass the argument now get different behavior.
Modified guard clauses. A guard clause that previously returned early on null now returns early on null or undefined. The behavior change is almost always harmless. In the one case where it matters, it is invisible.
Altered error handling. An exception that was previously propagated is now caught and logged. The caller that was handling the exception now never sees it. The error is silently swallowed.
Each of these categories deserves explicit attention in every refactor review. Spend more time on them than on the structural changes the refactor was supposed to make. The structural changes are the easy part. These are where the bugs live.
The Rewrite Anti-Pattern
Chapter 30 names this: the 500-line diff where every function was renamed and every variable was reorganized. Tests still pass. No one can find the three behavioral bugs.
This is the over-scoped refactor with insufficient verification. It is produced when the model is asked to “clean up this module” or “refactor this to be more readable” without scope constraints. The model does exactly what was asked. The result looks good. The behavioral delta is buried in the diff noise.
The protection is the combination of scope discipline and test coverage. If the diff is large, the scope was wrong. If the tests do not cover the changed behaviors, the verification cannot find the bugs. Both failures compound.
When a refactor diff arrives that is larger than expected, do not review it as written. Ask for it to be split. The split is not bureaucratic overhead - it is the mechanism that makes verification possible.
Practical Artifact - Refactoring Gate Checklist
| Gate | Action | What It Protects |
|---|---|---|
| Tests exist | Verify test coverage for code being refactored; write missing tests first | Ensures behavioral equivalence can be checked |
| Behaviors enumerated | List observable behaviors the code must preserve | Defines the scope of required verification |
| Trust boundaries enumerated | List schemas, validator order, authorization rules, state ownership, trace semantics, and rollback behavior the code touches | Defines what must survive besides behavior |
| Scope bounded | Define which files and functions are in scope; document it | Prevents the refactor from becoming a rewrite |
| Steps sized | Break the refactor into independently-verifiable steps | Keeps blast radius manageable |
| Suite passes at each step | Run tests after each intermediate step | Catches regressions immediately |
| Diff reviewed for behavioral change | Explicitly check renamed variables, changed defaults, modified guards, and altered error handling | Finds behavioral changes hiding in structural noise |
| Boundary-diff reviewed | For code touching truth, authorization, or state, confirm the same component owns it before and after | Catches trust-boundary moves invisible to behavioral tests |
| Full suite passes at completion | Run the complete test suite on the final state | Proves equivalence across all tested behaviors |
| Diff size reviewed | If the diff is unexpectedly large, split before merging | Prevents unverifiable rewrites from landing |
| Change type | Minimum required control |
|---|---|
| Rename variable or function | Type checker passes; tests pass |
| Extract function | Focused unit tests on the extracted behavior |
| Change module boundary | Integration tests covering the boundary |
| Change data shape | Schema validation and migration checks |
| Reorder validation, authorization, or state checks | Boundary-diff review naming the owner before and after |
| Replace deterministic logic with a model call | Not a refactor - requires acceptance criteria, a new validator, and review as a feature change |
| Rewrite section | Full behavioral specification and equivalence verification |
Export
Copy this block into your CLAUDE.md, agent instructions, or project checklist.
The safer the test harness, the more aggressive the refactor can be.
vcb_chapter: 21
title: "Refactoring With AI"
key_line: "The safer the test harness, the more aggressive the refactor can be."
thesis: "AI-assisted refactoring has a blast radius proportional to the diff size. Refactors must be reviewed for behavioral change, not just syntactic change."
checklist:
- item: "Write tests before starting any refactor on untested code"
protects: "Ensures behavioral equivalence can be verified"
- item: "Enumerate observable behaviors the refactor must preserve"
protects: "Defines what verification must cover"
- item: "Enumerate schemas, validator order, authorization rules, state ownership, trace semantics, and rollback behavior the code touches"
protects: "Defines what must survive besides behavior"
- item: "Document scope; reject changes outside it"
protects: "Prevents refactors from becoming rewrites"
- item: "Break into small, independently-verifiable steps"
protects: "Keeps blast radius manageable per step"
- item: "Run the full suite at each intermediate step"
protects: "Catches regressions before they compound"
- item: "Explicitly review renamed variables, changed defaults, modified guards, and altered error handling"
protects: "Finds behavioral changes hiding in structural noise"
- item: "Run a boundary-diff review naming the owner of truth before and after"
protects: "Catches trust-boundary moves invisible to behavioral tests"
- item: "Treat a deterministic-logic-to-model-call swap as a feature change, not a refactor"
protects: "Prevents an unreviewed relocation of a source of truth into the model's weights"
- item: "If the diff is unexpectedly large, split before merging"
protects: "Prevents unverifiable rewrites from landing"- Do tests exist for the code being refactored? - proves behavioral equivalence can be checked
- Are the observable behaviors the code must preserve written down? - defines required verification scope
- Are the trust boundaries the code touches written down - schemas, validator order, authorization, state ownership, trace, rollback? - defines what must survive besides behavior
- Is the refactor scope documented as a file and function list? - prevents scope expansion
- Is the refactor broken into steps that each pass the test suite? - keeps blast radius manageable
- Have renamed variables, changed defaults, modified guard clauses, and altered error handling been reviewed explicitly? - finds behavioral changes hiding in structural noise
- Has a boundary-diff review confirmed the same component owns truth before and after? - catches trust-boundary moves invisible to behavioral tests
- Does the full test suite pass on the final state? - proves equivalence across tested behaviors
- If the diff is larger than expected, has it been split? - prevents unverifiable rewrites from landing